Demonstrating, Evaluating, and Documenting Teaching Effectiveness Melissa S. Medina, Ed.D. Assistant Dean for Assessment & Evaluation and Presidential Professor University of Oklahoma College of Pharmacy AACP Institute May 22, 2013
Overview I.
Demonstrating effective teaching Scholarly Teaching & 6 standards of scholarly work
II.
Evaluating effective teaching (triangulate data) Rubrics for the 6 standards of scholarly work (self & peer) Teaching evaluation sample questions -6 standards (students)
III.
Documenting effective teaching Teaching philosophies & teaching philosophy rubric Teaching portfolios 2
1. Documenting Effective Teaching Scholarly Teaching 6 Standards of Scholarly Work
Demonstrate Teaching Effectiveness •
Traditionally demonstrated by
•
# of hours/courses taught Course evaluation results
Alternative way to demonstrate
Use scholarly teaching framework
4
Scholarly Teaching Defined Content knowledge
1.
Knowledge of subject area
•
Pedagogical knowledge
2. • •
Goes beyond delivering content Knowledge of how to teach subject, evidence-based • Preparing instructional/assessment materials • Making content understandable/simplified • Creating activities to reinforce learning • Motivating students
Shulman LS. Those who understand: Knowledge growth in teaching. Educational Researcher 1986;15 (2): 4-14.
Six Standards of Scholarly Work 1. 2. 3.
4. 5.
6.
Clear goals Adequate preparation Appropriate methods Significant results Effective presentation Reflective critique
Glassick CE, Huber MT, Maeroff GI. Scholarship Assessed: Evaluation of the Professoriate. San Francisco, CA: Jossey-Bass; 1997
Scholarship of Teaching & Learning •
Builds on process of scholarly teaching
•
Adds requirement of 3P’s
(Hutchings & Shulman, 1999)
1.
Makes scholarly teaching publicly disseminated
2.
Open to evaluation (peer-review)
3.
Presented in form others can build upon
Hutchings P, Shulman LS. The scholarship of teaching: New elaborations, new developments. Change. 1999;31:10-15.
Demonstrating Scholarly Teaching 6 Standards of Scholarly Work
1. Clear Goals •
Write and use effective learning objectives
•
Attend faculty development sessions about objective writing and use in teaching encounters
9
2. Adequate Preparation •
Use assessments to understand and describe students’ knowledge, skills and attitudes Uses results to create or modify teaching objectives, methods, materials, assessments, or style
•
Outline/map how your content fits into curriculum or course series
•
List types of technology used to deliver and/or assess their content Classroom management systems (e.g. Desire 2 Learn) Audience Response Systems 10
3. Appropriate Methods •
Uses and documents evidence-based strategies for teaching content
11
4. Significant Results •
Demonstrates alignment of objectives, teaching, and assessment
•
Writes valid and reliable test questions
•
Uses item analyses to improve tests & teaching
•
Creates rubrics & formative/summative assessment tools 12
5. Effective Presentation •
Presents learning outcomes to colleagues
•
Creates test banks
•
Receives peer evaluation of results
13
6. Reflective Critique •
Systematically reflects on outcomes using structured process or rubric
14
2. Evaluating Effective Teaching Triangulating teaching evaluation 6 standards rubrics (self & peer) Sample evaluation questions (students)
Triangulating Teaching Evaluations •
Aim is to make teaching evaluation more objective?
•
Triangulate teaching evaluations 1. 2. 3.
Self Peer Student
18
Self and Peer Evaluation • • •
See 6 Standards of Scholarly Teaching Rubrics in appendices Can be used for self-assessment Can also guide peers and focus their efforts during peer evaluation
19
Activity • •
Refer to any of the 6 rubrics and evaluate at least 1 area of your teaching material Pages 8-13 in handout
22
Student Evaluations of Teaching •
Consider aligning questions with 6 standards of scholarly teaching
23
1. Clear Goals •
The lecture objectives facilitated my learning (mark n/a if no objectives provided)
24
2. Adequate Preparation • •
The relevance of the instructor’s material was discussed The instructor reminded me of previous content that would help me learn the content he/she covered
25
3. Appropriate Teaching Methods •
The instructor used active learning in the course (e.g. activities, questions, teambased learning, cases, or discussions)
26
4. Significant Results •
The instructor used practice quizzes or clicker questions, cases, or other assessments throughout his/her teaching to assess my understanding
•
The instructor’s test questions and/or assessments were aligned with lecture objectives
•
Grading tools (such as rubrics, checklists, or rating scales) were used to grade my work
27
5. Effective Presentation •
The instructor's handouts/lecture notes helped me learn
•
The instructor used visual aids (such as slides) that helped him/her effectively explain the material
•
The instructor delivered engaging presentations
•
The instructor was available to answer questions 28
6. Reflective Critique •
The instructor encouraged me to reflect on my learning (of knowledge, skills or attitudes)
29
Activity •
Review the student evaluation of teaching questions used at your institution. 1.
2.
3.
How does your institution’s questions compare to the above samples? Determine if any of your questions could be aligned with the 6 standards of scholarly teaching. Consider other ways to gather feedback from students related to any or all of the 6 standards. 30
3. Documenting Effective Teaching Ongoing documentation Teaching Philosophy Teaching Portfolio
32
Documentation Tool/Strategy •
Teaching philosophy • Teaching portfolio • Annual report • Course evaluation summary table •
Course syllabi • Lecture materials • Examples of updated materials 33
Teaching Philosophy Documentation 9-item structure
9 Items to Address in Philosophy 1.
Start with an introduction.
Explain why teaching is your chosen career path. Explain what you like about teaching. Include an overview of teaching activities or an influential teaching encounter you have had.
9 Items to Address in Philosophy 2. Describe
your teaching belief(s)
You may have several beliefs in one teaching philosophy and may describe each separately
3. Explain
why that belief is important to help students learn 4. Cite literature that offers evidence of the benefit of your belief • Education research articles and theory • If you refer to specific article or readings include a reference and a reference list
9 Items to Address in Philosophy 5.
Describe how you enact that belief in your teaching or facilitate the learning process. What are teaching methods that help you realize this belief? Explain how have you improved or refined your strategy over time.
6.
What evidence have you collected regarding improved student learning outcomes? How do you assess student learning about this belief? Provide evidence or data you have collected about student learning outcomes related to this belief. 38
9 Items to Address in Philosophy A summary of feedback you have gotten from your learners about your ability to practice/achieve that belief “Comments I have received about my ability to do X are”: highlights the benefit of asking students for feedback about specific items in your teaching philosophy 8. Finish with a conclusion 9. References (if applicable) 7.
Activity 2 • •
Take 3 minutes Evaluate your own teaching philosophy using the teaching philosophy rubric
Handout page 14-15
Teaching Portfolio Documentation
Portfolio Strategies •
Use tables to quantify and allow for comparison across year
•
Teaching loads Course evaluations
See pages 16-17 in handout for sample tables
42
Summary •
Reviewed strategies for demonstrating, documenting, and evaluating teaching effectiveness Demonstrating – use scholarly teaching framework and 6 standards
Evaluating – Triangulate data using 6 standards rubrics and student evaluations Documenting – teaching philosophy 9-item structure (rubric) and teaching portfolio tables 43
Demonstrating, Evaluating, and Documenting Teaching Effectiveness Melissa S. Medina, Ed.D. Assistant Dean for Assessment & Evaluation and Presidential Professor University of Oklahoma College of Pharmacy AACP Institute May 22, 2013
