Testing the Consensus Model of Effective PD Analysis‐of‐Practice and the PD Research Terrain NARST 2015 Annual International Conference April 11-14, Hyatt Regency Chicago, Chicago, IL
Focus Questions for This Session • What is the STeLLA line of research, and what are we learning from this research? • How does the STeLLA line of research challenge and push beyond the consensus model of effective PD? • How can research on professional development produce stronger understandings of what makes for effective science teacher learning opportunities? © 2015 BSCS
STeLLA Line of Research Research on Student Ideas in Science
STeLLA I ViSTA STeLLA II EMAT RESPeCT
ViSTA Plus
© 2015 BSCS
Today’s Session • Introduction about STeLLA Line of Research: Kathy Roth, Cal Poly Pomona • The design and first test of STeLLA: Kathy Roth • Results from the CRT, scale‐up study of STeLLA: Chris Wilson, BSCS and Joe Taylor, Abt • Three new studies exploring scalability and sustainability of STeLLA: Connie Hvidsten, BSCS • STeLLA pushes beyond the consensus model of effective PD: Paul Numedahl, BSCS • Concluding comments: Kathy Roth • Discussant: Hilda Borko, Stanford University • Questions © 2015 BSCS
Additional authors • • • •
Jody Bintz, BSCS Betty Stennett, BSCS Susan Kowalski, BSCS, PI for EMAT Nicole Wickler, Cal Poly Pomona, PI for RESPeCT
© 2015 BSCS
Consensus Model of Effective PD (Yoon et al., 2007; Desimone, 2009; Wilson, 2013)
Effective PD … • Focuses on specific subject matter content • Engages teachers in active learning • Is coherent (aligned with teachers’ prior knowledge and beliefs; aligned with reform documents and school policy and practice) • Is of sufficient duration • Involves the collective participation of teachers (all teachers in a school, grade level, or dept.) © 2015 BSCS
Research supporting the consensus model • Relies heavily on teacher self‐report • Rarely looks at impact on student learning • And when the research does look at student learning, the results are mixed
© 2015 BSCS
Our Assertions • The consensus model is not enough to guide the development of professional development opportunities that will support NGSS. • Stronger lines of research on effective PD in science education are needed.
© 2015 BSCS
Science Teachers Learning from Lesson Analysis: STeLLA STeLLA is … • A year‐long, videocase‐based, analysis‐of‐ practice professional development program for upper elementary teachers • A line of research
© 2015 BSCS
STeLLA Line of Research Research on Student Ideas in Science
STeLLA I ViSTA STeLLA II EMAT RESPeCT
ViSTA Plus
© 2015 BSCS
What is the PD research terrain for the STeLLA line of research?
© 2015 BSCS
Mapping the PD Research Terrain (Borko, 2004) Phase 1. Existence Proof
Phase 2. Well‐specified PD Program
Phase 3. Multiple Programs
Single program
Single program
Multiple programs
Single site
Multiple sites
Multiple sites
Developers are PD providers New PD providers
New PD providers
Document teacher learning
Document student and teacher learning
Document teacher learning
© 2015 BSCS
The STeLLA PD Research Terrain Phase 1. Existence Proof and Quasi‐Experimental Design: STeLLA I STeLLA lesson analysis program is compared to business‐as‐usual Content Deepening program Single geographic area (southern California) Developers are PD providers Document student and teacher learning
© 2015 BSCS
The STeLLA PD Research Terrain Phase 1. Existence Proof and Quasi‐Experimental Design: STeLLA I
Phase 2. Well‐specified PD Program and Experimental CRT Design : STeLLA II
STeLLA lesson analysis program is compared to business‐as‐usual Content Deepening program
STeLLA Lesson Analysis program compared to well‐ specified Content Deepening program
Single geographic area (southern California)
New geographic area (Colorado Front Range)
Developers are PD providers New PD providers (BSCS) Document student and teacher learning
Document student and teacher learning
© 2015 BSCS
The STeLLA PD Research Terrain Phase 1. Existence Proof and Quasi‐Experimental Design: STeLLA I
Phase 2. Well‐specified PD Program and Experimental CRT Design : STeLLA II
Phase 3. Multiple Program Variations and Quasi‐ Experimental Designs: RESPeCT, ViSTA Plus, EMAT
STeLLA lesson analysis program is compared to business‐as‐usual Content Deepening program
STeLLA Lesson Analysis program compared to well‐ specified Content Deepening program
STeLLA Lesson Analysis program adapted for and tested in new contexts, with new target audiences
Single geographic area (southern California)
New geographic area (Colorado Front Range)
Multiple sites (CA, CO, TX, NM, online)
Developers are PD providers New PD providers (BSCS)
New PD providers
Document student and teacher learning
Document student and teacher learning
Document student and teacher learning
© 2015 BSCS
STeLLA Line of Research Research on Student Ideas in Science
STeLLA I ViSTA STeLLA II EMAT RESPeCT
ViSTA Plus
© 2015 BSCS
Mapping the PD Research Terrain: Existence Proof Phase 1, Borko
STeLLA I
Existence proof
Existence proof
Single program
Lesson analysis program compared to existing content deepening program of shorter duration
Single site
Single site (San Gabriel Valley, CA), four study groups
Developers are PD providers
Developers are PD providers
Outcomes focus: Teacher learning
Outcomes focus: Teacher learning, teaching practice, student learning Quasi‐experimental design
© 2015 BSCS
STeLLA Line of Research Research on Student Ideas in Science
STeLLA I ViSTA STeLLA II EMAT RESPeCT
ViSTA Plus
© 2015 BSCS
Mapping the PD Research Terrain: Testing a well‐specified program Phase 2, Borko
Phase 3, Borko
STeLLA II
Well‐specified program
Well‐specified programs
Well‐specified program
Single program
Multiple programs
Two programs
Multiple sites
Multiple sites
New sites (Front Range, CO), multiple lesson analysis study groups and content deepening groups
New PD leaders
New PD leaders
New PD leaders (BSCS)
Outcomes focus: Teacher learning
Outcomes focus: Teacher learning, student learning
Outcomes focus: Teacher learning, teacher practice, student learning Experimental design: RCT © 2015 BSCS
STeLLA Line of Research Research on Student Ideas in Science
STeLLA I ViSTA STeLLA II EMAT RESPeCT
ViSTA Plus
© 2015 BSCS
Mapping the PD Research Terrain: Adapting STeLLA for scalability and sustainability Phase 3, Borko
ViSTA Plus, EMAT, RESPeCT
Well‐specified programs
Adapting a well‐specified program for new audiences and for sustainability
Multiple programs
Multiple variations of the STeLLA program – variations in both substance and form
Multiple sites
Multiple sites, multiple audiences
New PD leaders
New PD leaders (including Teacher Leaders)
Outcomes focus: Teacher learning, student learning
Outcomes focus: Teacher learning, teacher practice, student learning Multiple quasi‐experimental studies
© 2015 BSCS
Mapping the PD Research Terrain: Adapting STeLLA for scalability and sustainability ViSTA Plus
EMAT
RESPeCT
Adapted STeLLA program for work with preservice ‐ first year teachers
Adapted STeLLA program for online work with HS teachers
Adapted STeLLA program for K‐6 teachers and students in a high‐needs urban district
Single 2‐year program compared to BaU
Single semester course, quasi‐experimental design
Single year‐long program compared to BaU
Two university sites
Online site
Pomona Unified School District, CA
New and experienced STeLLA PD providers
New PD provider
Teacher Leaders as PD providers
Outcomes focus: Teacher learning, teacher practice, student learning
Outcomes focus: Teacher Outcomes focus: Teacher learning, teaching practice, learning, student learning student learning © 2015 BSCS
Testing the Consensus Model of Effective PD: A Videocase‐based, Analysis‐of‐practice Program Kathleen Roth, Cal Poly Pomona
NARST 2015 Annual International Conference April 11‐14, Hyatt Regency Chicago, Chicago, IL
STeLLA Line of Research Research on Student Ideas in Science You are here!
STeLLA I ViSTA STeLLA II
EMAT
RESPeCT
ViSTA Plus © 2015 BSCS
Science Teachers Learning from Lesson Analysis (STeLLA) STeLLA I • An existence proof study • The PD program was designed, developed, and tested in a quasi‐experimental design
© 2015 BSCS
STeLLA Design Features • Engages teachers actively in collaborative, practice‐based inquiries • Treats content as central and intertwined with pedagogical issues • Enables teachers to see these issues embedded in real classroom contexts • Focuses on content and curriculum teachers will be teaching • Guided by an articulated model of teacher learning and clear teacher learning goals © 2015 BSCS
Describing the STeLLA Program • STeLLA theory of teacher learning • STeLLA program substance • STeLLA program form • STeLLA program resources © 2015 BSCS
STeLLA Theory of Teacher Learning • Situated cognition theory of teacher learning • Cognitive apprenticeship instructional model Program elements that embody this theory: 1) Learning progression starting with highly scaffolded experience moving towards greater independence 2) Experiences that create a “need to know,” dissonance 3) Use of experts who plan for and guide teacher learning © 2015 BSCS
STeLLA Program Substance: STeLLA Conceptual Framework
and STeLLA Teaching Strategies
© 2015 BSCS
© 2015 BSCS
STeLLA Program Substance: Teacher Learning Goals • Science content knowledge (CK) • Pedagogical content knowledge (PCK) related to the Student Thinking and Science Content Storyline Lenses • Ability to use CK and PCK to analyze teaching and learning • Ability to use CK and PCK in teaching science
© 2015 BSCS
STeLLA Program Form
Summer Institute 2 weeks (9 days) 58.5 hours face‐to‐face PD
Half‐day content deepening, led by university science faculty
Half‐day videocase‐based, lesson analysis sessions in grade‐level study groups of 5‐10 teachers, each led by a PD leader
© 2015 BSCS
STeLLA Program Form Fall • Grade‐level study groups, led by STeLLA PD leaders • Teachers analyze lesson video and Academic‐year ‐ student work from their teaching of the Monthly 3.5‐hour STeLLA lesson plans study group meetings Winter/Spring • Grade‐level study groups, led by STeLLA 30 hrs PD leaders face‐to‐face PD • In a second content area, teachers o analyze lesson video and student work from other teachers o plan lessons collaboratively © 2015 BSCS
STeLLA Program Form End of program
Teachers teach lessons they designed in the second content area
Total hours of face‐to‐face PD
88.5
© 2015 BSCS
STeLLA Program Resources
© 2015 BSCS
STeLLA Resources: Strategies Reading Booklet
© 2015 BSCS
STeLLA Resources: Videocases Video analysis tasks: Teachers analyze videocases from other teachers’ and each other’s classrooms Videocases include sequence of lesson videos student and teacher interview videos student written work student pretests and posttests © 2015 BSCS
STeLLA Resources: Analysis Process
FOCUS ON STUDENT THINKING AND SCIENCE CONTENT STORYLINE
© 2015 BSCS
STeLLA Resources: Analysis Guides SELECTING AND USING CONTENT REPRESENTATIONS Main learning goal:_____________________________________________________ ____________________________________________________________________ Description of content representation:______________________________________ ____________________________________________________________________ Part 1: Selecting the content representation: Is the content representation … YES
NO
1. Scientifically accurate? 2. Closely matched to the main learning goal? 3. Presenting science ideas in ways that are comprehensible to students? 4. Reinforcing or introducing student misconceptions? 5. Addressing common student misconceptions? 6. Distracting students from the main learning goal with too many details or new terms?
Part 2: Engaging students in using the content representation. Is the content representation used in a way that involves students in … YES
NO
1. Modifying or creating the content representation? 2. Analyzing the meaning of the content representation? 3. Critiquing the content representation?
Part 3: Suggestions for improvement_____________________________________
© 2015 BSCS
STeLLA Resources: Lesson plans
© 2015 BSCS
STeLLA Resources: Lesson Analysis Protocol
© 2015 BSCS
STeLLA Resources: Student pre‐post tests
© 2015 BSCS
STeLLA Resources: Features analysis charts
© 2015 BSCS
STeLLA Resources: Planning tools
© 2015 BSCS
STeLLA Resources: Content/PCK background readings
© 2015 BSCS
STeLLA Resources: Common Student Ideas
© 2015 BSCS
Testing the STeLLA Program Research Design: Quasi‐experimental 48 teachers and 1,460 students • Teachers self‐selected to Lesson Analysis (32) or Content Deepening Program (16) • Both groups received the same content deepening instruction in the summer • Lesson analysis teachers also spent time in lesson analysis in the summer and across the school year © 2015 BSCS
STeLLA Outcome Measures Teacher pre‐mid‐ post content knowledge test
Teacher pre‐mid‐ post video of science teaching
Teacher pre‐mid‐ post video analysis task
Student pre‐post content knowledge test
© 2015 BSCS
STeLLA I Results Outcomes
Results
Teacher Content Knowledge
Lesson Analysis > Content Deepening p Content Deepening p
