Measuring effectiveness of synchronous coaching using bug-in-ear device of pre-service teachers Coninx, N.S.
DOI: 10.6100/IR774337 Published: 28/05/2014
Document Version Publisher’s PDF, also known as Version of Record (includes final page, issue and volume numbers) Please check the document version of this publication: • A submitted manuscript is the author’s version of the article upon submission and before peer-review. There can be important differences between the submitted version and the official published version of record. People interested in the research are advised to contact the author for the final version of the publication, or visit the DOI to the publisher’s website. • The final author version and the galley proof are versions of the publication after peer review. • The final published version features the final layout of the paper including the volume, issue and page numbers.
Link to publication
Citation for published version (APA): Coninx, N. S. (2014). Measuring effectiveness of synchronous coaching using bug-in-ear device of pre-service teachers Eindhoven: Technische Universiteit Eindhoven DOI: 10.6100/IR774337
General rights Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights. • Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain • You may freely distribute the URL identifying the publication in the public portal ? Take down policy If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim.
Download date: 19. Jan. 2017
Measuring effectiveness of synchronous coaching using Bug-In-Ear device of pre-service teachers
Nele Sofie Coninx
This doctoral thesis was financially supported by Fontys Hogescholen and facilitated by the Eindhoven School of Education (Eindhoven University of Technology)
© 2014 Nele Sofie Coninx A catalogue record is available from the Eindhoven University of Technology Library ISBN: 978-90-386-3660-6 NUR: 841 Printed by: Digiking Hasselt Cover: Marie-Anne Dirkx
ii
Measuring effectiveness of synchronous coaching using Bug-In-Ear device of pre-service teachers
PROEFSCHRIFT
ter verkrijging van de graad van doctor aan de Technische Universiteit Eindhoven, op gezag van de rector magnificus prof.dr.ir. C.J. van Duijn, voor een commissie aangewezen door het College voor Promoties, in het openbaar te verdedigen op woensdag 28 mei 2014 om 16:00 uur
door
Nele Sofie Coninx
geboren te Hasselt, België
iii
Dit proefschrift is goedgekeurd door de promotoren en de samenstelling van de promotiecommissie is als volgt: voorzitter: 1e promotor: copromotor: leden:
adviseur:
iv
prof.dr. D. Beijaard prof.dr. W.M.G. Jochems dr.ir. C.J. Kreijns prof.dr. N. Engels (VU Brussel) prof.dr. J.M. Pieters (UT) prof.dr. K.A.H. van Leeuwen prof.dr. M.J. de Vries prof.dr. T.C.M. Bergen
Contents Voorwoord
ix
Chapter 1 Introduction
1
1.1 1.2 1.2.1 1.2.2
Background to the study Effective and efficient teacher trainings programs Coaching Feedback
1.3 The research questions References
1 3 3 4
5 6
Chapter 2 A review of technology-enabled immediate performance feedback 9 2.1 2.1.1 2.1.2 2.1.3 2.1.4
2.2 2.2.1 2.2.2 2.2.3 2.2.4
Introduction High rates of dropouts Effective and efficient teacher training programs Performance feedback Immediate feedback
Immediate performance feedback Method of analysis Who gave feedback to whom? What was this immediate performance feedback? How was the immediate performance feedback given?
2.3 A six-step approach to synchronous coaching 2.4 Conclusion and discussion References
Chapter 3 The use of keywords for delivering immediate performance feedback 3.1 3.2 3.3 3.3.1 3.3.2
Introduction Performance feedback Synchronous coaching Reflections on synchronous coaching Keywords for synchronous coaching
3.4 Method 3.5 Participants 3.6 Instruments 3.7 Results 3.8 Conclusion and discussion 3.9 Acknowledgement References
Chapter 4 Perceived satisfaction of pre-service teachers 4.1 4.2 4.2.1 4.2.2 4.2.3
9 9 11 11 12
13 13 20 21 25
27 29 31
37 37 38 39 40 41
42 44 45 46 48 50 50
55
Introduction Theoretical framework
55 56
Synchronous coaching Bug-in ear technology First level of reaction
56 57 58
v
4.3
Literature research
4.3.1 4.3.2
4.4
The research method Results
Research method
4.4.1 4.4.2 4.4.3 4.4.4
Participants Instrument Procedure Results
4.5 Conclusion and discussion References
Chapter 5 Observed behavioral change of pre-service teachers 5.1 5.2 5.3
Introduction Synchronous coaching Theoretical framework
5.3.1 5.3.2 5.3.3 5.3.4
5.4
Literature search Situational context of teacher behavior Operationalization of behavior Questionnaire on Teacher Interaction
Research method
5.4.1 5.4.2 5.4.3 5.4.4
Participants Instruments Procedure Analyses
5.5 Results 5.6 Conclusion and Discussion References
Chapter 6 Conclusion and discussion 6.1 6.2
Background to the study Brief overview of the research project
6.2.1 6.2.2 6.2.3 6.2.4
6.3
Synchronous coaching Cognitive load and ambiguity Perceived satisfaction of pre-service teachers Behavioral improvement of pre-service teachers
Limitations
6.3.1 6.3.2 6.3.3
6.4
Social validation Generalizability Opertionalizations of synchronous coaching
Suggestions for future research
6.4.1 6.4.2
Computer-provided immediate specific performance feedback Instrumentarium
6.5 Implications References
58 58 59
60 60 61 63 64
66 67
71 71 72 73 74 75 75 77
77 77 78 79 81
81 84 85
91 91 92 92 93 95 95
96 96 97 97
98 98 98
99 100
List of publications and presentations
103
Peer-reviewed articles Peer-reviewed presentations Professional articles and book (chapters) Professional presentations
103 103 103 104
Summary
vi
107
Samenvatting
111
Curriculum Vitae
115
Eindhoven School of Education Dissertation series
117
vii
viii
Voorwoord Tweeëntwintig jaar, pas afgestudeerd en dan sta je voor de eerste keer voor je eigen klas. Welgeteld achtentwintig paar ogen van beroepsleerlingen kijken toe hoe jij hip, cool en trendy een theoretisch vak probeert over te brengen. Een pittige opdracht. Gelukkig kon ik rekenen op de ondersteuning van mijn mama die in dezelfde school lesgaf. Wanneer een les een totaal andere (lees foute) wending aannam, spurtte ik naar haar lokaal om een luisterend oor te vinden. Meerdere keren heeft ze meewarig gefluisterd dat ze me haar ‘trukendoos’ wilde doorgeven. Nu, meer dan tien jaar later, snap ik wat ze toen bedoelde. Met trukendoos bedoelde mijn mama, dat er effectieve gedragingen zijn, die een leraar gebruikt in zijn les. Zo bijvoorbeeld is het hebben van oogcontact en het ‘even zwijgen’ en rondkijken in de klas, een voorbeeld van gedragingen die aan orde houden kunnen bijdragen. Stel je nu eens voor dat een beginnende leerkracht, net op het moment wanneer je die gedragingen het best toepast, hier feedback over zou kunnen krijgen… onmiddellijke, on-thespot feedback als het ware. Synchroon coachen is de methodiek die dit toelaat en hierover gaat dit proefschrift. Voor dit promotie-onderzoek hebben vele leraren en leraren-in-opleiding me toegelaten in hun wereld. Dankzij hun bijdrage aan het onderzoek heb ik de inzichten op het gebied van synchroon coachen kunnen aanscherpen en een verbeterde, effectievere manier van coachen kunnen ontwikkelen gesteund op wetenschappelijke evidentie. Op deze manier hebben de coaches in samenwerking met hun leraren-in-opleiding bijgedragen aan het ontstaan van dit proefschrift. Onze dank gaat daarom uit naar directies, leraren, begeleiders-op-school, leraren-in-opleiding en opleidingsdocenten die wilden meewerken aan het onderzoek. We denken hierbij voornamelijk aan Trevianum (Sittard), Connect College (Echt), Kwadrant (Weert), Philips van Horne (Weert), Grotius College (Heerlen), Stella Maris College (Meerssen), Pleincollege Eckart (Eindhoven), Lorentz Casimir Lyceum (Eindhoven), Dr. Knippenbergcollege (Helmond), Newmancollege (Breda), Bernardinuscollege (Heerlen) en Strabrecht College (Geldrop). Het promoveren is nooit makkelijk geweest. Maar dan toch, het zit erop, mijn proefschrift is klaar. In ruim vier jaar heb ik ontzettend veel geleerd, zowel op professioneel als op persoonlijk vlak. Ik ben als onderzoeker en als mens gegroeid. Nu, leren doe je nooit alleen en een aantal mensen wil ik hiervoor bedanken. Allereerst mijn promotor, Wim Jochems. Veel dank voor de manier waarop je me heb begeleid tijdens mijn promotietraject. Je was kritisch en trok dikwijls aan de teugels, iets wat ik zeker op het einde nodig had. Ik heb veel van je geleerd, zeker op methodologisch vlak. Toen ik mama werd, kregen onze gesprekken een andere wending. Je kon er van genieten dat jouw kleinzoon sneller liep dan mijn dochter Zana, maar je kon er nog meer van genieten dat Zana sneller kon praten. Karel, zonder jou, zou het boekje er niet liggen. Maar jou teleurstellen is nu eenmaal geen optie. Als dagelijkse begeleider van de enige vrouwelijke FLOS (Fontys Lerarenopleiding Sittard) promovenda had je het niet altijd makkelijk. Toch stond je deur altijd letterlijk en figuurlijk open. Je was mijn klankbord. De voortgang van het onderzoek was het meest besproken onderwerp in onze wekelijkse sessies. Maar ook bespraken we de perikelen van FLOS, mijn loopbaan en thuissituatie. Ik kreeg van jou het onmisbaar vertrouwen met bijhorende professionele ruimte om me te ontwikkelen, waarvoor dank. Op onderzoek matig vlak had je steeds in een recordtempo al mijn stukken grondig bestudeerd, wat diepgaande interessante discussies opleverde. Je dacht daarbij pragmatisch, waarbij je de grote lijnen en
ix
structuren hebt aangereikt. Toch blijf het hoogtepunt van de afgelopen jaren de trip naar de BETH conferentie te London, waar ik met een wijselijke glimlach aan terug denk. Voor de prettige samenwerking en de altijd weer leuke momenten dank ik mijn FLOScollega’s. Wat een voorrecht om op deze manier onderzoek te mogen doen. Jullie vierden de successen met me mee en steunden me als het minder ging. In het bijzonder denk ik dan aan mijn collega’s van de kenniskring Educatieve functies van ICT (Guido, Alexandra, Maurice, Frank, Paul, Gerard, Jan, Niek, Pierre en Ton). Heel erg bedankt. Ik ga jullie missen. Ook mijn collega’s van de lerarenopleiding Pedagogisch Technische Hogeschool Eindhoven die aan de zijlijn meekeken en die steeds ondersteunend meedachten, ben ik dank verschuldigd. Een speciale dank gaat naar Paul Dirckx voor zijn immer stabiele aanwezigheid in de synchroon coachen wereld, telkens een stapje vooruit denkend, kennis en passie combineren met relativerende inzichten. Ik heb heel veel van jou geleerd. Mijn collega’s van de Eindhoven School of Education hebben opbouwend kritische commentaar geleverd op de wetenschappelijke stukken. Bedankt voor jullie inspirerende en constructieve ideeën. Met veel plezier denk ik terug aan de lunchpauzes, cursussen en congressen. Speciale dank aan de ESoE nieuwsbrief-redactie Anna, Fons en Zeger-Jan. Medepromovendi Wietse, Rens, Marieke, Linda, Niek en Evelien (a.k.a. knappe blonde) wil ik ook bedanken, het was gezellig om samen te werken en we hebben heel wat van elkaar geleerd. Evelien en Wietse promoveerden dankzij een dijk aan doorzettingsvermogen. Ook Marieke en Linda haalden de eindstreep. Rens en Niek zitten ons op de hielen. De Leuvengroep (en ja ik noem jullie bij naam: Geertrui, Stephanie, Hannelore, Kelly, Bo, Kristien, Jurgen, Tim, Wonne, Maxime, Stijn, Thomas en knappe Swiffer) biedt een bijzondere invulling aan mijn donderdagavond waar er vanalles ter tafel komt, behalve synchroon coachen. Waarvoor dank! Claudia en Bart, jullie zijn vrienden voor het leven. En natuurlijk het thuisfront. Mijn relativerende schoonfamilie, dit is wat ik vier jaren gedaan heb. Mama en papa, ik denk dat jullie wel fier zijn. Nooit opgeven heb ik van jullie geleerd. Mama, jij was het meest kritisch, ik hoop ‘iets goeds’ voor de kinderen gedaan te hebben. Zus, jij startte als eerste met je promotie en zei dat dat wel iets voor mij is. Je blijft ook nu weer de slimste van ons twee. Vooral dank voor de extra hulp bij de afronding. Rutger zit daar ook wel voor iets tussen… Een speciaal woord van dank is voor Jo’ke. Je betekent ontzettend veel voor me en je bent er altijd als ik je nodig heb. Dank je wel voor je niet aflatend geduld me laten ‘tetteren’ en je relativeringsvermogen om de zaken in het juiste (humoristisch) perspectief te zien. Jij en Zana, geven aan elke dag een gouden randje mee. Diepenbeek, daar waar mijn zonnetjes schijnen. Nele Coninx
x
Dit proefschrift is opgedragen aan alle leraren die ondanks synchroon coachen leerlingen begeleiden naar hun volwassenheid. [Dit woordje van dank heb ik neergeschreven in mijn thesis over aanvangsbeleiding van leraren (2001) en vind ik ook hier van toepassing.]
xi
xii
Chapter 1 Introduction
Ministers of Education of the European Union in 2007 agreed that (EC Counsil, 2007, p.2): ‘High quality teaching is a prerequisite for high-quality education and training, which are in turn powerful determinants of Europe’s long-term competitiveness and capacity to create more jobs and growth in line with the Lisbon goals ...’ This has led the Council in November 2009 agreeing on the professional development of teachers and school leaders, that (EC Counsil, 2007, p.3): ‘In view of the increasing demands placed upon them and the growing complexity of their roles, teachers need access to effective personal and professional support throughout their careers, and particularly during the time they first enter the profession. [...] Efforts should be made to ensure that all newly qualified teachers receive sufficient and effective support and guidance during the first few years of their careers.’ If, then, the quality of teachers is so important, we should take care that teachers will receive all the support enabling them to achieve these high levels, especially in the teacher training phase. Facing the issue of teacher shortages, teacher preparation and induction has gained increased attention as a means of assisting pre-service teachers’ successful entry into the teaching profession (Giebelhaus & Bowman, 2002).
1.1 Background to the study Teachers are identified as an important factor influencing the quality of education in schools (Hattie, 2003). In becoming an expert teacher their professional development consists of three stages, which all teachers go through. The first stage is the initial teacher training where they master the qualifications needed to become a teacher. The second stage is the induction stage where the teacher develops into an independent teacher by going through the first years of confrontation with reality, often experienced as a ‘reality shock’ which can take place during a training program or in-service during the first years of teaching. The third stage is the continuing professional development stage. Those teachers have overcome the initial challenges of becoming a teacher. The quality of the development of these teachers depends on the support that is given in each of these three stages (EC Directorate-General for Education and Culture, 2010). Therefore, it is important to improve and sustain the quality of the teacher by improving preservice teacher preparation. It is expected that such efforts will deliver well-prepared new teachers who easily can handle all kinds of potential problems they will meet in the classroom. Nevertheless, the teaching profession shows alarmingly high rates of dropouts during the induction phase. Many newly qualified teachers leave the job early (De Jonge & De Muijnck, 2002). Some studies have found that as many as 50% of new teachers leave within the first 5 years of entry into the occupation (Ingersoll & Smith, 2004). According to 1
Chapter 1 another study 10% of the newly qualified teachers quit the job after the first year of teaching and after five years the number increases to 25% (Van Kregten & Moerkamp, 2004). The present research aims at offering an approach that could rereduce the dropout rate. There are several causes for these dropout rates. New teachers mention a number of reasons for leaving the profession after their first year (Boe, Cook & Sunderland, 2008; Kersaint, Lewis, Potter & Meisels, 2007; Smith & Ingersoll, 2004). These reasons are related to the (lack of) support they have got during their induction phase and can be categorized and elaborated according to 1) the characteristics of the profession, 2) the emphasis of the preparation before beginning to teach and 3) the consequences of this preparation. Characteristics of the profession Teaching has become more demanding: teachers now have to deal with a greater diversity of children; the increased use of technology in schools, the many innovations in schools, new teaching methods based on constructivist pedagogy, and so on. The diversity of children with different cultural and socioeconomic backgrounds has grown and children with different cognitive abilities are now allowed in the same class (integration of special education in ‘normal ‘schools). Beginning teachers have consistently reported feeling unprepared and experiencing a lack of support during their teacher training phase to cope with students’ diverse backgrounds and fail to facilitate effective instructional strategies designed to support students from diverse backgrounds (Boe, Cook & Sunderland, 2008; Ingersoll & Smith, 2004). Emphasis on preparation Traditionally teacher preparation is isolated from what is happing in reality in schools. Only at times when student teachers have an internship in schools, they can see these schools from the inside and form an idea how teaching is in reality. However, this internship last only a few weeks and is too short to experience how teaching actually is. Also, students are protected in these internships to be involved in problematic or complex situations and thus these students practice often in ‘safe’ environments. In short, teacher preparation curricula follow the so called Cartesian model (i.e. the separation of the learner form the learning environment, that is, the learner is supposed to learn in a safe environment rather than in the raw reality of the school) (Barab & Plucker, 2002). As a consequence, when these student teachers graduate and get a job, they may experience a ‘reality-shock.’ This experience is assumed to be a result of the limited practical relevance of teachers’ training programmes (Crasborn & Hennissen, 2010). In the nineties, this was seen as a result of the gap between initial teacher education and the job itself (De Vries, Beijaard & Buitink, 2008). Consequences The high fall-out rates costs schools and teacher preparation programs time and money as they continually must recruit and prepare new educators (Duke, Karson, & Wheeler, 2006). Also, high rates of turnover of induction programs contribute significantly to the attrition of beginning teachers in education (Wade, 2010). To avoid dropout rates there has to be a focus on teacher preparation. Teacher training should depart from the Cartesian model and embrace ecological approaches (Barab & Plucker, 2002). Such an ecological approach is realized when schools are seen as learning environments and as partners in the teacher preparation programs. Besides, policy-makers have the idea that teacher education is less expensive if it is done in the workplace (Caldwell & Carter, 1993). Therefore, the teacher training phase is considered a part of the teacher’s development, whether or not it takes place within the first years of employment. The ‘teacher training’ therefore forms the departing point of the present study.
2
Introduction And because of the turn-over issues, the research has turned on to the effectiveness and efficiency of this training. However, rather than that in the present study evidence is gathered that an improved training is reducing dropout rates, it is assumed that effective and efficient programs will lead to higher quality teachers and that this will, in turn, will impact dropout rates negatively (i.e., they will go down). The aim of this PhD research is to concentrate on an effective training program as synchronous coaching and the aim is to measure the effectiveness of synchronous coaching using Bug-in-Ear-technology (BIE-technology). The general research question is “What are the effects of synchronous coaching of pre-service teachers?” We will elaborate it in greater detail in section 1.3 of this chapter.
1.2 Effective and efficient teacher trainings programs The objective of professional teacher training programs is to improve the performance and retention of pre-service teachers. This will enhance and prevent the loss of investments in teacher’s human capital. We know a good deal about the characteristics of successful professional development (Ingersoll & Strong, 2011). A particularly critical component of teacher preparation programs is an internship within a classroom (Darling-Hammond & Cobb, 1996; Scheeler, McAfee, Ruhl, & Lee, 2006). The best way a pre-service teacher can learn the knowledge, skills and attitudes which are the components of the profession of a teacher is in the actual professional practice. In the professional practice the pre-service teacher experiences all kinds of lifelike situations which a beginning teacher should master in front of the class. In the confrontation with complex, lifelike practice problems, the pre-service teacher starts to develop and adapt the competencies which are necessarily for the practice (Putnam & Borko, 2000). Therefore, investing in the competence development during teacher training programs seems an effective strategy to provide students with what we called “the basic things” (Veenman, 1984; Houtveen, Versloot, & Groenen, 2006). Conducting research on the effectiveness and efficiency of an teacher training program of pre-service teachers requires a definition of what an effective and efficient tacher training program is and contains. The most effective and essential elements of a program are identified in research published over the last two decades. This research yielded several key elements that must be used in the support of pre-service teachers (Billingsley, Griffin, Smith & Kamman, 2009; Ingersoll & Kralik, 2004). Two variables that have the most impact on teacher retention are defined, namely coaching and feedback (Smith & Ingersoll, 2004; Darling-Hammond, Wei, Andree, Richardson, & Orphanos, 2009; Feiman-Nemser, 2001). We will discuss successively these characteristics. 1.2.1
Coaching
In order to master new teaching strategies and skills effectively teachers must receive coaching while applying what they are learning. Coaching preferably involves an expert in the area, but can also be provide by a peer. Most important is that the teacher receives feedback on his teaching behavior in order to get adequate information about quality and adequacy of that behavior. Coaching is the process of being observed during teaching, usually by a coach
3
Chapter 1 or a peer, and receiving feedback on the teaching behavior. This helps the teacher internalize what is being learned through observation and feedback (Cooper, 2004). Coaching is based on a collegial relationship in which the coach helps the pre-service teacher through formal and informal interactions. A characteristic of coaching is that a pre-service teacher gets support while learning to achieve a specific result or goal. The pre-service teachers may be in an intern or apprenticeship relationship with the experienced coach. Coaching not only asks for a lot of experience as a teacher, but also the ability to communicate adequately. Learning and implementing new instructional strategies requires not only technical expertise, but also the ability to decide when to use particular strategies and how to modify them so they optimize learning. Coaching has been utilized in variety of settings, but the goal is always the same: to improve instruction for all children (Glickman, Gordon, & Ross-Gordon, 1995). Pre-service teachers are engaged in job-embedded conversations aimed at improving instructional practices (Wade, 2010). 1.2.2
Feedback
In general terms, feedback is any message generated in response to a learner’s action. Preservice teachers should receive explicit guidance and constructive feedback on their performance within classrooms, or they may practice ineffective strategies that they might continue later during their teaching careers. Coaches can provide feedback in two ways. In deferred feedback, coaches take narrative or quantitative data while observing a lesson and share those data with the teacher upon lesson completion, perhaps even later. In immediate feedback, supervisors observe a problem in a teacher behavior, or lack thereof, instantly stop the instructional activity and correct the teacher’s behavior directly during the observation (Scheeler et al., 2006). According to the first way an experienced teacher provides feedback to pre-service teachers in a deferred manner, as the observer attempts to remain unobtrusive and silent while taking notes in an effort to avoid interruption of the instructional flow in the classroom (Scheeler et al., 2006; Scheeler & Lee, 2002; West & Jones, 2007). One shortcoming of this method pertains to providing feedback after the fact, which results in new teachers being susceptible to using teaching methods that may be incorrect and inaccurate (Colvin, Flannery, Sugai, & Monegan, 2009; Scheeler & Lee, 2002; Scheeler, 2008; Scheeler, Bruno & Grubb, 2009). In the second way, this shortcoming is taken away but sadly has introduced a new problem/shortcoming: in order to immediately correct teacher’s behavior there is no other way than to stop instantly the instructional activity. This introduces a disturbance in the classroom as the students/pupils are left on there own during the time the coach is giving feedback to the new/pre-service teacher. The ability of coaches to observe pre-service teachers and provide immediate feedback through the use of Bug-In-Ear (BIE) technology offers a solution to this latter problem/shortcoming. Using BIE device new/pre-service teachers are provided with instant, in-flight feedback, allowing immediate correction of teacher behaviors without interruption or disturbance (Colvin et al., 2009; Coulter & Grossen, 1997; Giebelhaus, 1994; Goodman, Brady, Duffy, Scott & Pollard, 2008; Rock, Gregg, Thead, Acker, Gable & Zigmund, 2009; Scheeler et al., 2009). BIE device is a tiny ear piece that is used in for example cycling sports. An advantage of using immediate performance feedback using BIE device is that it provides learners with an opportunity to change target behaviors while practicing a skill, as opposed to 4
Introduction their repeating errors until feedback is delivered, after the fact (Coulter & Grossen, 1997). Herold, Ramirez & Newkirk found in 1971 the use of BIE device to be promising. Research indicates that immediate performance feedback using BIE allows the pre-service teacher to immediately correct his or her behavior and to practice the correct behavior, which provides for more practice opportunities (Mallott & Suarez, 2004). Although there has been some research involving the effect of immediate performance feedback using BIE device on pre-service teachers, there is limited research regarding the effects of immediate performance feedback using BIE device (through technology) and the ability to increase specific teacher behaviors identified as effective in the classroom. One might, for example, argue that receiving immediate feedback is producing cognitive overload. The teacher has to manage the classroom and at the same time to process the feedback. And how to avoid ambiguity in the feedback? Besides, there is some evidence that teachers’ appreciation and acceptation of feedback varies which seems related to specific personality characteristics (Hooreman, 2008). Thus, there is a need to investigate the potential effect of providing immediate feedback to beginning teachers through BIE support to improve the instructional delivery and teacher feedback to students.
1.3 The research questions This dissertation addresses the improvement of coaching of pre-service teachers and examines the effectiveness of synchronous coaching of pre-service teachers with BIE device, as a means to support pre-service teachers with immediate performance feedback. Effectiveness is in this study defined in comparison with the traditional method of coaching the pre-service teacher. The overall research question is: “To what degree is synchronous coaching using BIE device effective in terms of behavioral change in the proposed desired direction (= becoming more cooperative and having more ascendancy in the pre-service’s class room)?” In order to address the general research question, four more specific research questions were derived: This results in the different research questions: a) Which systematic approach is suitable for immediate performance feedback using BIE device of pre-service teachers? (Chapter 2) b) Does this systematic approach reduces cognitive load and ambiguity (Chapter 3) c) To what degree is the six steps (systematic) approach effective for perceived satisfaction of pre-service teachers? (Chapter 4) d) To what degree is the six steps (systematic) approach of synchronous coaching effective for behavioral improvement of pre-service teachers (Chapter 5)? Different factors contributing to the effectiveness of synchronous coaching using BIE device were first investigated by deriving a six step approach. Therefore, Chapter 2 presents a literature review of synchronous coaching for defining a consistent method for effective synchronous coaching is presented. The research resulted in a six step approach for synchronous coaching describing a procedure for applying BIE device adequately. Next, the innovative use of keywords was measured so to reduce cognitive load and ambiguity. Chapter 3 describes how the problems of cognitive load and ambiguity are tackled. The systematic approach is tested in an in vivo pilot test to see if it reduces cognitive load and ambiguity of 5
Chapter 1 pre-service teachers during the teacher training. Next two in depth investigations have been conducted to measure the effectiveness of synchronous coaching. In Chapter 4 the effectiveness of the method is research by determining the perceived satisfaction of preservice teachers on the functions of coaching. It is shown what the satisfaction on supportiveness, giving feedback, discussing suggestions, discussing effects on pupils and encouragement is, when synchronous coaching with the use of BIE device. In Chapter 5, the effects of synchronous coaching with the use of BIE device and a six step approach on the behavior of pre-service teachers is determined in an in vivo experiment. Finally in Chapter 6 the results of the studies described in the previous chapters are summarized for each of the research questions. This is followed by a general discussion of the findings, a number of practical implications, some limitations of the study and suggestions for further research. Table 1-1: Overview of the dissertation Chapter 1. General introduction
Research questions Problem statement: The improvement of coaching of pre-service teachers by examing the effectiveness of synchronous coaching using BIE device.
2. A review of technology-enabled immediate performance feedback 3. The use of keywords for delivering immediate performance feedback 4. Perceived satisfaction of synchronous coaching using BIE device 5. Perceived behavioral change of synchronous coaching using BIE device 6. General conclusions and discussions
Which systematic approach is suitable for immediate performance feedback using BIE device of pre-service teachers? Does this systematic approach reduces cognitive load and ambiguity? To what degree is the six steps (systematic) approach effective for perceived satisfaction of pre-service teachers? To what degree is the six steps (systematic) approach of synchronous coaching effective for behavioral improvement of pre-service teachers? Main conclusions of the research project and discussion of the various findings.
References Barab, S.A., & Plucker, J.A. (2002). Smart people or smart contexts? Cognition, ability, and talent development in an age of situated approaches to knowing and learning. Educational Psychologist, 37(3), 165–182. Billingsley, B.S., Griffin, C.C., Smith, S.J., Kamman, M., & Israel, M. (2009). A review of teacher induction in special education: Research, Practice, and Technology Solutions, NCIPP Document No. RS-1. Boe, E.E., Cook, L.H., & Sunderland, R.J. (2008). Teacher turnover: Examining exit attrition, teaching area transfer, and school migration. Exceptional Children, 75(1), 7-31. Caldwell, B.J., & Carter, E.M.A. (1993). The Return of the Mentor: Strategies for Workplace Learning. The Falmer Press, London. Crasborn, F.J.A.J. & Hennissen, P.P.M. (2010). The skilled mentor: Mentor teachers’use and acquisition of supervisory skills. Unpublished doctoral thesis. Eindhoven, The Nederlands: Eindhoven School of Education. Colvin, G., Flannery, K.B., Sugai, G., & Monegan, J. (2009). Using observational data to provide performance feedback to teachers: A high school case study. Preventing School Failure, 53(2), 95-104. Cooper, J.D. (2004). Professional Development: An Effective Research-Based Model. Houghton-Mifflin Harcourt Professional Development.
6
Introduction Coulter, G.A., & Grossen, B. (1997). The effectiveness of in-class instructive feedback versus after-class instructive feedback for teachers learning direct instruction teaching behaviors. Effective School Practices, 16, 21–35. Darling-Hammond, L., & Cobb, V.L. (1996). The changing context of teacher education. In F. B. Murray (Ed.). The teacher educator’s handbook: Building a knowledge base for the preparation of teachers. 14-62. San Francisco, CA: Jossey Bass. Darling-Hammond, L., Wei, R.C., Andree, A., Richardson, N., & Orphanos, S. (2009). Professional Learning in the Learning Profession: A Status Report on Teacher Development in the United States and Abroad. Palo Alto, CA: School Redesign Network at Stanford University. De Jonge, J.F.M. & Muijnck, J.A. de (2002). Waarom leraren de sector verlaten. Onderzoek naar de uitstroom in het primair en secundair onderwijs.[Why do theachers leave their profession? Research of drop-outs in primary and secondary education].The Nederlands, Zoetermeer: EIM. Duke, L., Karson, A., & Wheeler, J. (2006). Do Mentoring and induction programs have greater benefits for teachers who lack pre-service training? Journal of Public and International Affairs, 17(2), 61-82. EC Counsil (2007), Conclusions of the Council and of the Representatives of the Governments of the Member States, meeting within the Council of 15 November 2007, on improving the quality of teacher education (Official Journal 2007/C 300/07 of 12.12.2007). EC Directorate-General for Education and Culture (2010), Developing coherent and systemwide induction programmes for beginning teachers: a handbook for policymakers, European Commission, SEC(2010)538 final. Feiman-Nemser, S. (2001). From preparation to practice: Designing a continuum to strengthen en sustain teaching. Teacher College Record, 103, 1013-1055. Giebelhaus, C.R. (1994). The bug-in-the-ear Device: An anlternative Student Teaching Supervision Strategy, paper presented at the association of teacher educators’ annual conference, Atlanta, GA. Giebelhaus, C.R. & Bowman, C.L. (2002). Teaching mentors: Is it worth the effort? The Journal of Educational Research, 95 (4), 246-254. Glickman, C.D., Gordon, S. P., & Ross-Gordon, J.M. (1995). Supervision of instruction (3rd ed.). Needham Heights, MA: Simon & Schuster. Goodman, J.I., Brady, M.P., Duffy, M.L., Scott, J., & Pollard, N.E. (2008). The effects of ‘bug-in-ear’ supervision on special education teachers’ delivery of learn units. Focus on autism and other developmental disabilities. 23 (4), 207-216. Hattie, J. (2003). Teachers make a difference, What is the research evidence? University of Auckland, Australian Council for Educational Research. Herold, P., Ramirez, M., & Newkirk, J. (1971). A portable radio communication system for teacher education. Educational Technology, 11, 30-32. Hooreman, R.W. (2008). Synchronous Coaching of Trainee Teachers, an experimental approach (ICO Dissertation Series No. 188). Unpublished doctoral thesis. Eindhoven, The Nederlands: Eindhoven School of Education) Houtveen, T., Versloot, B., & Groenen, I. (2006). De begeleiding van beginnende leraren. [Coaching of beginning teachers]. Utrecht, Sectiebestuur Onderwijsarbeidsmarkt (SBO). Retrieved from http://www.onderwijsarbeidsmarkt.nl/fileadmin/user_upload /begeleiding_startende_leraren.pdf. Ingersoll, R., & Kralik, J. (2004). The impact of mentoring on teacher retention: What the research says. Denver, CO: Education Commission of the States.
7
Chapter 1 Ingersoll, R. & Smith, T.M. (2004) Do teacher induction and Mentoring Matter? NAASP Bulletin, 88(638), 28-40. Ingersoll, R., & Strong, M. (2011). The Impact of Induction and Mentoring Programs fro Beginning Teachers: A Critical Review of the Research. Review of Educational Research, 81(2), 201-233. Kersaint, G., Lewis, J., Potter, R., & Meisels, G. (2007). Why teachers leave: Factors that influence retention and resignation. Teaching and Teacher Education, 23(6), 775-794. Kregten, A. v., & Moerkamp, T. (2004). Startende leraren in het buitenland. Begeleiding van beginnende leraren in Canada, Engeland, Frankrijk, de Verenigde Staten en Zwitserland. [Beginning teachers abroad. Coaching of beginning teachers in Canada, England, France, United States and Switzerland]. Den Haag: SBO. Mallott, R.W., & Suarez, E.T. (2004). Principles of behavior (5th ed.). Upper Saddle River, NJ: Pearson/Prentice Hall. Putnam, R.T. & Borko, H.(2000).What do new views of knowledge and thinking have to say about research on teacher learning? Educational Researcher, 29(1), 4-15. Rock, M.L., Gregg, M., Thead, B.K., Acker, S.E., Gable, R.A., & Zigmond, N.P. (2009). Can you hear me now? Evaluation of an online wireless technology to provide real-time feedback to special education teachers-in-training. Teacher Education and Special Education, 32(1), 64-82. Scheeler, M.C., & Lee, D.L. (2002). Using technology to deliver immediate corrective feedback to preservice teachers. Journal of behavioral education, 11(4), 231-241. Scheeler, M.C., McAfee, J.K., Ruhl, K.L., & Lee, D.L. (2006). Effects of corrective feedback delivered via wireless technology on preservice teacher performance and student behavior. Teacher education and special education, 29(1), 12-25. Scheeler, M. C. (2008). Generalizing effective teaching skills: The missing link in teacher preparation. Journal of Behavioral Education, 17, 145-159. Scheeler, M. C., Bruno, K. & Grubb, E. (2009). Generalizing Teaching Techniques from university to K-12 classrooms: teaching pre service to use what they learn. Journal of Behavioral Education, 18, 189-210. Smith, T.M., & Ingersoll, R.M. (2004). What are the effects of induction and mentoring on beginning teacher turnover? American Educational Research Journal, 41(3), 681-714. Veenman, S. (1984). Perceived problems of beginning teachers. Review of Educational Research, 54(2), 143-178. Vries, S. de, Beijaard, D., & Buitink, J. (2008). Learning in the context of ‘co-creation of educational pratices’ In: Ax, J. & P. Ponte (Eds.), The teaching profession in Dutch educational praxis, Sense Publishers,163-181. Wade, Y.W. (2010). Increasing novice teacher support in 21st century classrooms: Induction and Mentoring for beginning teachers through bug-in-ear technology. Unpublished doctoral thesis. Orlando, Florida, ; University of Central Florida. West, E., & Jones, P., (2007). A framework for planning technology used in teacher education programs that serve rural communities. Rural Special Education Quarterly, 26(4), 315.
8
Chapter 2 A review of technology-enabled immediate performance feedback1 Abstract: Performance feedback in teacher preparation and in the workplace learning of preservice teachers is increasingly becoming an important issue with respect to the enhancement of the quality of teachers. Enhancing the quality of teachers is necessary in light of current teachers’ shortages. A systematic literature study was performed to examine the differences and commonalities between various forms of technology-enabled immediate feedback. Immediate performance feedback is usually accomplished by a bug-in-ear (BIE). This article presents the results of the literature review and proposes a six-step approach to synchronous coaching. Also, we delineate the characteristics/ attributes of effective synchronous coaching, which can be used in actual situations in the class. Keywords: performance feedback, immediate feedback, BIE device, pre-service teachers, synchronous coaching
2.1 Introduction Teachers are identified as an important factor influencing the quality of education in schools (Hattie, 2003). The professional development of teachers consists of three stages, which all teachers go through: the initial teacher education, the induction phase, and the continuing professional development phase. The quality of the development of these teachers depends on the support that is given in each of these three stages (EC, 2010). 2.1.1
High rates of dropouts
The teaching profession has alarmingly high rates of dropouts during the induction phase. A lot of newly qualified teachers leave the job early (De Jonge & De Muijnck, 2002). A number of studies have found that as many as 50% of new teachers leave within the first five years of entry into the occupation (Smith & Ingersoll, 2004). According to a different study, after the first year of teaching, 10% of newly qualified teachers quit their teaching jobs, and after five years, the number increases to 25% (Kregten & Moerkamp, 2004). There are several causes for these fall-out rates. New teachers indicated as reasons for leaving the profession after their first year (Boe, Cook, & Sunderland, 2008; Kersaint, Lewis, Potter & Meisels, 2007; Smith & Ingersoll, 2004): 1) not having a mentor from a similar field of study; 2) lack of common planning times with mentor teachers; and 3) limited opportunity for mentor feedback on field observations of 25 beginning teachers. 1
This chapter is submitted for publication as an article titled “A review of technology-enabled immediate performance feedback”.
9
Chapter 2
These overall reasons, which are related to the support that they receive during their induction phase, can be categorized and elaborated according to the characteristics of the profession, the emphasis of the preparation before beginning to teach, and the consequences of this education. Teaching has become more demanding. The diversity present within student populations presents new challenges for teachers. Children come to school from various cultural and socioeconomic backgrounds, each bringing different kinds of experiences to school. In addition, laws mandate the inclusion of children with special needs into regular classroom settings. This means that all teachers must become more acquainted with effective instructional methods for exceptional children and work together to support student learning. Beginning teachers have consistently reported feeling unprepared and that they have a lack of support during their teacher training phase. These teachers reported struggling with students’ diverse backgrounds and fail to facilitate effective instructional strategies designed to support students from diverse backgrounds (Boe et al., 2008; Ingersoll & Smith, 2004). The increasing criticism of the relevance of theory in teacher education as a preparation for practice has become apparent (Hennissen, Crasborn, Brouwer, Korthagen, & Bergen, 2008). The so-called reality shock experienced when confronting a classroom for the first time is assumed to be a result of the limited practical relevance of teachers’ training programs (Crasborn, 2010). In the 1990s, this was seen as a result of the gap between initial teacher education and the job itself (Vries, Beijaard, & Buitink, 2008). In practice, teacher training is common, but tacher training that is intensive, comprehensive, structured, and sequentially delivered in response to teachers’ emerging pedagogical skills and knowledge is not. Preservice teachers are often left on their own to succeed or fail in their classrooms, an experience that contains the feeling of being lost (Ingersoll, 2012). Studies have shown that pre-service teachers can experience feelings of anxiety, isolation, powerlessness, and a lack of support (Brown, 2005; Rushton, 2003; Woullard & Coats, 2004). Research and development of an effective teacher training program to reduce the reality shock is needed. The high fall-out rates cost schools and teacher preparation programs time and money, as they continually must find and prepare new educators (Duke, Karson, & Wheeler, 2006). Also, high rates of turnover of induction programs contribute significantly to the attrition of beginning teachers in education (Wade, 2010). Some schools attempt to stem the tide of new teacher attrition through comprehensive teacher training programs that include mentoring, reacting upon the issues cited by departing teachers as the cause of their dissatisfaction (Duke et al., 2006). Facing the issue of teacher shortages, teacher preparation and induction has gained increased attention as a means of assisting pre-service teachers’ successful entry into the teaching profession (Giebelhaus & Bowman, 2002). Supporting and optimizing teacher professional development more systematic and empirically validated research is needed (Hasweh, 2003). Policymakers have the idea that teacher education is less expensive if it is done in the workplace (Caldwell & Carter, 1993). High levels of turnover are costly in both obvious and less visible ways. Among these costs is the current teacher shortage (Ingersoll, 2001). So, not only effective, but also efficient, teacher training programs are the ultimate goal (Wade, 2010).
10
A review of technology-enabled immediate performance feedback 2.1.2
Effective and efficient teacher training programs
Researchers have focused on efficient and effective means of preparing pre-service teachers who can reach the higher standards imposed by the changing society and also remain in the teaching profession (Coonen, 2005). Preparing pre-service teachers for a diverse population of pupils is an important task that must be addressed systematically. Because society requires accountability measures, educators must ensure that pupils achieve at high levels. Current teacher education literature addresses the need for standards and accountability and stresses the importance of well-researched, teacher preparation methods (Goodman & Duffy, 2007). This article provides a teacher preparation method that is based on the implications of the literature of coaching and feedback. Coaching is an effective way of providing support to pre-service teachers while improving teachers’ instructional effectiveness (Fienem-Nemser, 2001). Scheeler, McAfee, and Ruhl (2004) determined that teacher instruction improved when a coach provided specific, goaloriented, and immediate performance feedback to pre-service teachers. Teachers, as other learners, acquire and maintain new behaviors best when they receive systematic instruction; have multiple opportunities to practice; and receive performance feedback that is specific, goal-oriented, and immediate. If professionals in the field of teacher preparation strive to improve the effectiveness of teachers, they should seek ways to implement the identified attributes in an efficient and consistent way. 2.1.3
Performance feedback
If coaching is to train pre-service teachers to behave more productively in their classroom, their performance must be measured in the classroom to determine if the desired changes do, in fact, occur and must be set as criteria for teachers to obtain (Bowles & Nelson, 1976). Coaching is the process of giving feedback on performance. Ramaprasad (1983) defined performance feedback as “externally given information about performance, which is used to reduce the gap between the actual knowledge level and the reference knowledge level” (p.8). In general terms, feedback is any message generated in response to a learner’s action. Supervisors can provide feedback in two ways. In deferred feedback, supervisors take narrative or quantitative data while observing a lesson and share those data with the teacher upon lesson completion, perhaps even later. In immediate feedback, supervisors observe a problem in a teacher’s behavior, or lack thereof, instantly stop the instructional activity, and correct the teacher’s behavior directly during the observation (Scheeler, McAfee, Ruhl, & Lee, 2006). Scheeler and others (2004) demonstrated that the quality of the performance feedback increased if this feedback was specific and goal-oriented. Feedback needs to be specific, instead of general. Furthermore, goal-oriented feedback (feedback that identifies the type and extent of errors, as well as provides specific ways to correct them; see also below) is purported to be one of the most useful tools for improving well-learned errors because it demonstrates correct responses (Scheeler et al., 2004). An unambiguous measure of performance development clearly needs to be included in future evaluations of coaching (White & Gordon, 2000). This is referred to as goal-oriented feedback (Shute, 2008). Goal-oriented feedback was a key element in enhancing learning.
11
Chapter 2 Judgments are made about the match between evidence of achievements and standards (Knight, 2002) and linked criteria that sets targets for improvement (Sadler, 1989, 1998). Indicators for pre-service teachers for performance feedback should be fine-tuned (Goodman, Brady, Duffy, Scott, & Pollard, 2008). This enables the coach to address a single teaching behavior, making it possible to focus on the quality of the teaching exchange (Scheeler et al., 2006). 2.1.4
Immediate feedback
Immediate feedback was defined as feedback delivered by the coach within three seconds after the target behavior occurred (Scheeler, Macluckie, & Albright, 2008; Scheeler & Lee, 2002). This criterion was established in a separate study that looked at the time between a feedback message and the response to this message (Scheeler & Lee, 2002). In immediate teaching, the coach notes a teacher’s problem of strength, immediately stops instruction, and explains the error or models an alternative for the pre-service teacher. Immediateness is identified as an effective attribute of feedback. It was demonstrated that teachers acquired targeted teaching behaviors faster and more efficiently when feedback was immediate. Pre-service teachers were able to learn how to teach more in less time (efficiency of learning), and they were able to model effective instruction techniques (Scheeler et al., 2004). There are different ways to give immediate feedback. One way is that a coach can interrupt or even stop the lesson and give specific performance feedback. This method has obvious disadvantages, such as a decrease of concentration and potentially a reduction of the selfesteem of the pre-service teacher. Furthermore, because of the disruption, there is valuable lesson time lost (Scheeler et al., 2006). Additionally, immediate feedback enables the coach to address a single behavior, making it possible to focus on the quality of the teaching exchange; however, immediate feedback may interrupt the flow of instruction and distract the student’s attention from the task or teacher (O’Reilly, Renzaglia, & Lee, 1994). The challenge for coaches is to find a way to give immediate performance feedback that is not intrusive, wherein lesson momentum is maintained and the student’s academic learning time is increased. A possible solution lies in the technical advances that allow coaches to deliver immediate performance feedback without interruption, such as BIE (Scheeler et al., 2004). The major advantage of BIE is the immediacy of feedback to the teacher and the flexibility of the instruments as compared with other devices, such as light or noise generators (Bowles & Nelson, 1976). In this article, we present our systematic literature study. This study concentrated, in particular, on immediate feedback. In order to be unobtrusive, technology, such as a bug-inear (BIE) device, is used in immediate feedback. We have discussed the characteristics/attributes of immediate performance feedback. Based on the literature study, we end with a definition of synchronous coaching, which is a six-step method for delivering immediate performance feedback to pre-service teachers by using a BIE device.
12
A review of technology-enabled immediate performance feedback
2.2 Immediate performance feedback Immediate performance feedback provides pre-service teachers with immediate, on-the-spot feedback while these pre-service teachers are engaging in classroom instruction (Giebelhaus, 1994). Although researchers have used various forms of technology to provide immediate feedback, the bug-in-ear (BIE) is widely the most frequently used device. BIE device is referring to a small ear bud receiver that transmits verbal communication from a transmitter to a receiver. Researchers had variously referred to this technology as a whisper in my ear (WIME), wireless earphone, a mechanical third ear device, or an electronic audio-cueing system (Rock, Gregg, Thead, Acker, Gable, & Zigmond, 2009). Most of us know BIE device as the tiny earpieces that people wear while they talk on their cell phones, instead of having to hold the phone to their ear. 2.2.1
Method of analysis
We performed a SCIENCE search using a combination of the following terms: “immediate feedback,” “performance feedback,” “bug-in-ear,” “wireless communication,” and “teachers.” We selected certain articles if they included one or more of these terms. Articles including the above search terms combinations in their title or abstract and focusing on immediate feedback were included in the overview. The “snowball” method was applied by examining the used references of the articles, which led to other meaningful articles. Also, conference papers and other documents that colleagues sent to us were also included if they were relevant for our study. The raw yield was 61 articles. These were further examined on additional criteria, including the clarity of the described method section and the use of an actual BIE. Studies where the researchers used immediate e-mail after the lessons (Rathel, Drasgow, & Christle, 2008) or researchers who used keyword entries placed on a monitor (Neukrug, 1991) were excluded because of the lack of the use of BIE. Thus, excluded sources were either off topic or poorly designed. On the basis of these criteria, the net yield was 29 articles. The articles that remained can be found in Table 2-1. 2.2.1.1 General overview Based on these 29 articles, a general inventory of characteristics of the research described is compiled to obtain insight into the validity of the articles. The characteristics used are the context or domain of the research described, the manner of data collection, the number of subjects, and the method of research. This led to a matrix, which is shown in Table 2-1. Most of the studies (23 of 29 studies) have been carried out in the education sector, with 49% in primary education, 26% in special education, and 17% in middle school education. Also, the majority of the studies (18 of the 29 studies) reported qualitative data. Twelve of the 29 studies used statistical analyses and get therefore the term quantitative data. Of the 29 studies, there are only 8 studies that used an experimental comparison between a control group and an experimental group. The numbers of respondents are relatively low, going from a single subject to 34 respondents. White and Gordon (2000) are an exception, with a study of 60 respondents.
13
Chapter 2 2.2.1.2 Research questions The scope of this article is to find an overview of who, what, and how immediate performance feedback is used. We analyzed 29 articles to provide answers to the following questions: 1) 2) 3) 4) 5) 6)
Who gave the immediate performance feedback? What was this immediate performance feedback? How was the immediate performance feedback given? Where was the immediate performance feedback given? Which BIE was used to give immediate performance feedback? Was there specific training necessary to give immediate performance feedback?
These three main questions were successively answered by analyzing these 29 articles; in the analysis process, we coded the data that was presented in the articles (Field, 2009). In Table 2-2, Table 2-3, Table 2-4, and Table 2-5, the coded data is represented and will be elaborated upon in the next paragraphs. The reliability was sufficient. We reference to Field (2009). In this article, the same technique and reliability are established. A junior researcher was trained on coding procedures. The researchers used a point-by-point analysis for one randomly selected article with 90% agreement between the researcher and the junior researcher. No additional training was further needed on coding articles. Three other articles were randomly selected for coding. Inter-rater agreement data were collected on three of all of the 29 articles. The researcher and the junior researcher coded these articles separately. At the end of the coding, the researcher calculated a total agreement on the rate of specific statements made between the researcher and junior researcher. Article 1 had an inter-rater reliability with a mean of 98% agreement between coders. There was 100% agreement for article 2, and article 3 had a mean of 98% agreement between the two coders.
14
A review of technology-enabled immediate performance feedback Table 2-1: General overview literature search Article Reference
Research Data collection
Context or domain
Method of research Number of subjects Experimental
Quantitative
3
x
x
13
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x x
x
x
x
x
x
x
x
x x
x
x
x
12
x
1
x
20
x
8
x
21 22
x
x
1
x
3
x
x
x
x
x
x
x
x
30
x x
x x
x
x
34 2 30
x
1
x
x x
22
x
x
x
x
x
x
x
25
x
x
Qualitative
N
Audio taped
x
Observations
Audio taped
x
University
Elementary - primary
Special
Middle school
Observations
Education
Counseling Bennett, Brady, Scott, Dukes, Frain (2010) Bowles, Nelson (1976) Chow, Mak, Cheung, Louie (2001) Crimmens, Bradlyn, StLawrence (1984) Domoto, Weinstein, Getz (1979) Farrell, Chandler (2008) Fry, Hin (2006) Giebelhaus, Cruz (1994) Giebelhaus, Cruz (1995) Giebelhaus (1994) Goodman, Duffy (2007) Goodman, Brady, Duffy, Scott, Pollard (2008) Gordon, Kogan (1975) Hunt (1980) Kahan (2002) Kogan, Gordon, Wimberger (1972) Price, MarchandMartella (2002)
x
x x x
x
x
x
15
Chapter 2 Article Reference
Research Data collection
Context or domain
Method of research Number of subjects Qualitative
Quantitative
Experimental
N
Audio taped
Observations
Audio taped
Observations
Education
Counseling
University
Elementary - primary
Special
Middle school
Rock, Gregg, x x x 15 x Thead, Acker, Gable, Zigmond(2009 Scheeler, Lee x x x 3 x (2002) Scheeler, x x x 5 x McAfee, Ruhl, Lee (2006) Scheeler, x x x 4 x Macluckie, Albright (2008) Scheeler, Bruno, x x x 3 x Grubb (2009) Silverman, x x 16 x x Kimmel (1972) Thomson (1978) x x 8 x Van der Mars x x 1 x (1988) Ward (1960) x 21 x Ward (1962) x 20 x White, Gordon x x 60 x (2000) Wimberger, x x 1 x x Kogan (1974) (*) CT (cooperative teacher), M (Mother), NT (novice teacher), O (observer), P (pupil), PST (pre-service teacher), ST (student teacher), S (Student), R (researcher), T (teacher), US (university supervisor), W (worker)
16
A review of technology-enabled immediate performance feedback Table 2-2: Who has given immediate performance feedback to whom? Article Reference
Content of immediate performance feedback Who has given immediate performance feedback to whom (*) Coached Coach Coach experienced? W R x
Bennett, Brady, Scott, Dukes, Frain (2010) Bowles, Nelson (1976) T R x Chow, Mak, Cheung, Louie ST ST (2001) Crimmens, Bradlyn, StM R x Lawrence (1984) Domoto, Weinstein, Getz (1979) S US Farrell, Chandler (2008) PST CT x Fry, Hin (2006) ST ST Giebelhaus, Cruz (1994) ST CT x Giebelhaus, Cruz (1995) ST US-CT x Giebelhaus (1994) ST CT x Goodman, Duffy (2007) P T x Goodman, Brady, Duffy, Scott, NT R x Pollard (2008) Gordon, Kogan (1975) M R x Hunt (1980) ST R x Kahan (2002) ST CT x Kogan, Gordon, Wimberger M R x (1972) Price, Marchand-Martella P R x (2002) Rock, Gregg, Thead, Acker, T R x Gable, Zigmond(2009 Scheeler, Lee (2002) PST S x Scheeler, McAfee, Ruhl, Lee PST R x (2006) Scheeler, Macluckie, Albright ST ST (2008) Scheeler, Bruno, Grubb (2009) PST R x Silverman, Kimmel (1972) PST PST Thomson (1978) PST CT x Van der Mars (1988) T R x Ward (1960) S R x Ward (1962) S T x White, Gordon (2000) S S Wimberger, Kogan (1974) M R x (*) CT (cooperative teacher), M (Mother), NT (novice teacher), O (observer), P (pupil), PST (pre-service teacher), ST (student teacher), S (Student), R (researcher), T (teacher), US (university supervisor), W (worker)
17
Chapter 2 Table 2-3: What was this immediate feedback? Article Reference
Content of immediate performance feedback What was this immediate feedback? Valence of feedback
Content of feedback
x x
x
x x x
x x x
x
x
x x x x x x x
x x
x x x x
x x
x x
x x
Descriptive
x x x x x x
Prescriptive
x x x
x x x x x x
Interrogative
x
x
x
x
x
x x x
x x x
x x x
x x x
x
x x x x x x x x
x x x x x x x x
x
x x x x x x x x
x x x x x x x x
x
x x x x x x
x x x x x x
x x x
x x x x x x
x x x
x x
x x x x x x
x x x x
x x x x
x x x x
x x x x
x x x x x x
x x x x x x
General
x
x
Specific
x
x x
Positive
x
Corrective
x
Type of feedback
Neutral
Management
Instruction
Pedagogical
18
Other
Non verbal behavior Bennett, Brady, Scott, Dukes, Frain (2010) Bowles, Nelson (1976) Chow, Mak, Cheung, Louie (2001) Crimmens, Bradlyn, St-Lawrence (1984) Domoto, Weinstein, Getz (1979) Farrell, Chandler (2008) Fry, Hin (2006) Giebelhaus, Cruz (1994) Giebelhaus, Cruz (1995) Giebelhaus (1994) Goodman, Duffy (2007) Goodman, Brady, Duffy, Scott, Pollard (2008) Gordon, Kogan (1975) Hunt (1980) Kahan (2002) Kogan, Gordon, Wimberger (1972) Price, Marchand-Martella (2002) Rock, Gregg, Thead, Acker, Gable, Zigmond(2009 Scheeler, Lee (2002) Scheeler, McAfee, Ruhl, Lee (2006) Scheeler, Macluckie, Albright (2008) Scheeler, Bruno, Grubb (2009) Silverman, Kimmel (1972) Thomson (1978) Van der Mars (1988) Ward (1960) Ward (1962) White, Gordon (2000) Wimberger, Kogan (1974)
Precision of feedback
x
x
x x
x
x
x
x x x x
x x x
x
x x x x
x x x x x
x x
x
x x x
A review of technology-enabled immediate performance feedback Table 2-4: How did they give immediate performance feedback? Article Reference
Content of immediate performance feedback How did they give immediate performance feedback? Used technology Setting Protocol
x x x x
x x
x x
x
x x x x x x
x x x x x x
x x x x x
x x x x x x x x x x x x x x
x
x x x
x
x
x
x
x x x x x x
x
x x x x x x x
x x x
x x
x x x x x
x x x x x x
x x x x x x
x x x x x x x x x x
x x x x x x x x x x x
x x
Prerequisites
x
x x x x x x x
x x x x x x x x x x x x
Trainings program
x
x x x x x x
x
x x x x
Baseline
x x x x
x x
Intervention
x x
x
x
Acquisition
x x
x x x x
Maintenance data
x
x x x x x
Experimental
x x x
Natural
x
Ear piece
x x
x x x x x x x x
One way radio
x x
Two way radio
In the room (visual and sound) Adjacent room (visual no sound) Different room (no visual-sound) Bennett, Brady, Scott, Dukes, Frain (2010) Bowles, Nelson (1976) Chow, Mak, Cheung, Louie (2001) Crimmens, Bradlyn, St-Lawrence (1984) Domoto, Weinstein, Getz (1979) Farrell, Chandler (2008) Fry, Hin (2006) Giebelhaus, Cruz (1994) Giebelhaus, Cruz (1995) Giebelhaus (1994) Goodman, Duffy (2007) Goodman, Brady, Duffy, Scott, Pollard (2008) Gordon, Kogan (1975) Hunt (1980) Kahan (2002) Kogan, Gordon, Wimberger (1972) Price, Marchand-Martella (2002) Rock, Gregg, Thead, Acker, Gable, Zigmond(2009 Scheeler, Lee (2002) Scheeler, McAfee, Ruhl, Lee (2006) Scheeler, Macluckie, Albright (2008) Scheeler, Bruno, Grubb (2009) Silverman, Kimmel (1972) Thomson (1978) Van der Mars (1988) Ward (1960) Ward (1962) White, Gordon (2000) Wimberger, Kogan (1974)
x x x
x x x x
x x
x x
x
x
x x
x x x x x x x x x x x
x x
x x
x
x x x
x x x
x x x x x
19
Chapter 2 Table 2-5: Effects of immediate performance feedback Article Reference
Effects What were the effects of immediate performance feedback? Perceived/ observed?
x
x
x x x x x x x
x
x
x x
x x x
x x x x x x
x x x
x x
x x x x x
x
x x x x x
x x x x x x x x x x x x
x x x
x x x x x x x x x x
x x x x x
x x x
x x
Coach
x
Coached
Learning
x x
Effects for who? Reaction
Behavior
x x
Results
2.2.2
Observed
Perceived
Bennett, Brady, Scott, Dukes, Frain (2010) Bowles, Nelson (1976) Chow, Mak, Cheung, Louie (2001) Crimmens, Bradlyn, St-Lawrence (1984) Domoto, Weinstein, Getz (1979) Farrell, Chandler (2008) Fry, Hin (2006) Giebelhaus, Cruz (1994) Giebelhaus, Cruz (1995) Giebelhaus (1994) Goodman, Duffy (2007) Goodman, Brady, Duffy, Scott, Pollard (2008) Gordon, Kogan (1975) Hunt (1980) Kahan (2002) Kogan, Gordon, Wimberger (1972) Price, Marchand-Martella (2002) Rock, Gregg, Thead, Acker, Gable, Zigmond(2009 Scheeler, Lee (2002) Scheeler, McAfee, Ruhl, Lee (2006) Scheeler, Macluckie, Albright (2008) Scheeler, Bruno, Grubb (2009) Silverman, Kimmel (1972) Thomson (1978) Van der Mars (1988) Ward (1960) Ward (1962) White, Gordon (2000) Wimberger, Kogan (1974)
Effects on what?
x x x
x x x
x x x x x x x x x x x x x x x x x x x
Who gave feedback to whom?
Of the 23 studies that were conducted in the education, there are 15 studies that gave immediate performance feedback to pre-service teachers (65%). Also immediate performance feedback is given to students/pupils (22%) and to experienced teachers (13%). This is not surprisingly because pre-service teachers (Fry & Hin, 2006) or teachers (Goodman et al., 2008) who experience more difficulties in their classroom, will profit the most of immediate performance feedback. Should a coach have significant experience, or could peers also provide immediate performance feedback? Three quarter of the studies stated that a coach should have experience in the performance that he will give immediate feedback on. Although White and Gordon (2000) stated through observations that the quality of the immediate performance feedback did not increase when it was given by peers, peers themselves stated that they were not so satisfied with their role of a coach. Another study of Chow and colleagues (2001) confirmed that some pre-service teachers were averse in giving immediate feedback to peers 20
A review of technology-enabled immediate performance feedback because it felt like they were criticizing their fellow peers. Other researchers (Scheeler et al., 2004) reported that giving immediate performance feedback by peers could let to a number of disadvantages: 1) a necessary training for peers with associated costs; 2) a guarantee of consistency when peers are included in the process of evaluations of fellow peers; and 3) the high demands that are made to peers in giving immediate feedback on behavior, which they, themselves, should learn. The quality of immediate performance feedback, therefore, depended on the experience of the coach and, in particular, the degree of experience in the specific skills on which immediate performance feedback was given. The more experienced a coach was in a certain skill, the more assured and the more specific the coach was in giving immediate performance feedback (Chow et al., 2001). For this reason, Giebelhaus and Cruz (1992) stated that immediate performance feedback should be given by an experienced coach, instead of a peer. Only in that way could constructive and meaningful feedback be given. Who should this experienced coach be? Of the 17 studies in which an experienced coach had been used, eight studies reported the use of a researcher as a coach. In seven studies, the experienced coach was a cooperating teacher, and in two studies, the experienced coach was a teacher educator. Price and colleagues (2002) stated explicitly that the experienced coach should not be a researcher. Farrell and Chandler (2008) confirmed this and reported that a cooperating teacher is most suitable in giving immediate performance feedback to pre-service teachers. A cooperating teacher spends a large part of his time in coaching pre-service teachers. Cooperating teachers are in the ideal position to observe the performance of preservice teachers and to give immediate performance feedback. This way, a cooperating teacher has more time to give feedback because he can also give feedback while the preservice teacher is teaching. 2.2.3
What was this immediate performance feedback?
Kahan (2002) made a framework to classify immediate performance feedback. He distinguished the immediate performance feedback messages into type (descriptive, corrective, or interrogative), precision (general or specific), valence (encouraging, corrective, or neutral) and content (classroom management, instruction, pedagogical, or other). We used his framework to classify immediate performance feedback messages. Several researchers used (a part of) a similar classification (Rock et al., 2009; Scheeler et al., 2004; Giebelhaus, 1994; Ward, 1960). 2.2.3.1 Type of immediate performance feedback An immediate performance feedback message was coded as follows: 1) “descriptive,” as the contents of a feedback message mainly described a teaching situation; 2) “corrective” if a pre-service teacher explicitly was told to take specific actions; and 3) “interrogative” if the contents of the feedback message were presented as a question.
21
Chapter 2 During the same lesson, corrective, descriptive, and interrogative feedback messages can be used. Twenty-five of the 29 studies used corrective feedback. Using corrective immediate performance feedback had two advantages. Pre-service teachers were made conscious about a certain performance, and corrective feedback indicated clearly which performance alternative could serve as a solution. Twelve studies used descriptive feedback, and in only two studies, questions were asked as an immediate performance feedback. Rock and colleagues (2009) confirmed that corrective feedback was used most in giving immediate feedback messages by means of a BIE device, and interrogative feedback was given less. Several researchers confirmed that the use of corrective immediate performance feedback is more useful (Scheeler et al., 2004; Giebelhaus, 1994). 2.2.3.2
Precision of immediate performance feedback
An immediate performance feedback message was coded as follows: 1) “general” if the message gives indications or if the message described performance in general terms that a teacher showed; and 2) “specific” if the message was strongly aimed at a specific aspect of performance so that the message was immediately understood. Of the 29 studies, 26 studies used specific performance feedback. In six studies, general performance feedback was used. This is in line with the feedback literature. Feedback is specific when it provides information about a particular aspect of performance (Shute, 2008). The feedback should be directed at the task (Black & Wiliam, 1998) and detailed (Ross & Tronson, 2005). Research findings demonstrated that when the pre-service feedback is focused, specific, clear, and obtainable, the learning to teach is more effective (Peard & Hudson, 2006). The research findings also demonstrated that specific feedback resulted in positive changes in teacher performance (Scheeler et al., 2004). According to Locke and Latham (1984), specific feedback was more effective than general feedback, primarily because it focused on the pre-service teachers’ attention, resulting in the fact that the feedback could be more directed. Specific feedback could also be made more manageable by narrowing the range of possible interpretations of the feedback, reducing misunderstanding (Sweller, 1990). Possible meanings of a feedback message could cause ambiguity, resulting in higher levels of cognitive load. Ambiguity and cognitive load could lead to lower levels of learning (Kluger & DeNisi, 1996; Sweller, Van Merriënboer, & Paas, 1998) or even reduced motivation to respond to the feedback (Ashford, 1986). 2.2.3.3
Valence of immediate performance feedback
An immediate performance feedback message was coded as follows: 1) “encouraging,” if appreciation was expressed for a demonstration of the specific teacher performance; 2) “corrective” if a message was given to recognize the impact of a pre-service teacher’s performance by offering an alternative to perform; and 3) “neutral” if a message was expressed in a neutral tone.
22
A review of technology-enabled immediate performance feedback Of the 29 studies 25 used corrective feedback messages. These messages have been expressed in a neutral manner (this is true for all of the 25 studies). Moreover, in 16 studies, there was explicit expression of encouraging feedback messages. Rock and colleagues (2009) confirmed that corrective and encouraging feedback were used most in giving immediate performance feedback messages. White and Gordon (2000) stated that the immediate feedback messages should be expressed in an encouraging and not criticizing manner. Wimberger and Kogan reported in 1974 that encouraging immediate feedback messages are prerequisites for a pre-service teacher to give attention to corrective and specific feedback messages. 2.2.3.4
Type + precision + valence of immediate performance feedback
From the literature study, we found that corrective feedback was always very specific. When giving immediate performance feedback with a BIE, the feedback message was corrective (giving a certain alternative for performance) and specific (discussed in advance and including a well-defined alternative for performance). In most studies, corrective feedback the coach and the pre-service teacher discussed this corrective feedback in advance, and this feedback was aimed at a specific performance. Some researchers used the term “code system” (Farrell & Chandler, 2008), in which they discussed in advance what was meant with a certain “cod.” This code acted as a summary of the specific, corrective, immediate performance feedback message, and these codes should be displaced in an encouraging manner (Rock et al., 2009). These are in advance-structured feedback messages. These in advance-structured feedback messages were short but clear (Thomson, 1978). Goodman and colleagues (2008) stated that besides restricting the amount of messages, the messages should also be predictable and should be used only in targeted lessons. Moreover, the appropriateness of the coach intervention can largely be averted if the coach and the preservice teacher agree in advance on the performance objectives that they both wish to achieve (Herold, Ramirez, & Newkirk, 1971). That way, feedback messages are more meaningful for the pre-service teacher (Chow, Mak, Chung, & Louie, 2010), and it enables the coaches to cue on performance outside of the target area (Giebelhaus & Cruz, 1994). The use of in advance-structured messages has another advantage. Too-frequent messages disrupt the pre-service teacher’s train of thoughts (Hunt, 1980). However, the use of short, specific, in-advance, structured messages may have been helpful in preventing disruptions in the flow of instruction (Scheeler & Lee, 2002). The coach should restrict the length and frequency of his messages as a means of keeping nuisance and confusion factors at a minimum (Herold et al., 1971). Our explanation is that the confusion was caused because at the moment of the actual communication, two information sources competed for the attention of the pre-service teacher, the first being the actual classroom (visual and auditory information), and the second being the coach (using an BIE device: auditory information); thus, longer messages meant a longer staying in this state. In the cognitive load theory (Sweller, 1988; Sweller et al., 1998), the division of attention could produce cognitive (over)load (Chandler & Sweller, 1992). Cognitive load is defined as “cognitive capacity that is actually allocated to accommodate the demands imposed by the task” (Paas, Tuovinen, Tabbers, & Van Gerven, 2003, p. 64).
23
Chapter 2 To ensure the use of in advance-structured, encouraging, immediate performance feedback, we propose that the feedback messages should be pre-recorded as audio files that could be played on the spot. For example, each feedback message can be a small audio tape (of an encouraging voice) and could be displayed on a laptop. The pronouncement of a feedback message could be activated by a mouse click (on a laptop). Because the feedback messages were pre-defined and audiotaped in advance, the coach is enabled to “whisper” other messages than the constructed set. Such a computer program can also make logs of the used feedback messages. For research purposes, the consistent use of immediate performance feedback is guaranteed. Other problems reported in the literature could also be solved. Coaches should be instructed to speak softly and positively when giving prompts for encouragement, reinforcing feedback, or corrective feedback so that a supportive tone is delivered (White & Gordon, 2000). The coaches should not have to learn to adjust their voice quality (Farrell & Chandler, 2008), timing, and language (Chow et al., 2001). It is no longer critical that performance feedback should be made positive in tone and content (Price et al., 2002). This allows the coach to prompt only proportions of the strategy when needed (Goodman & Duffy, 2007), and the size of room and the acoustics (Giebelhaus & Cruz, 1992) should not play a determining role. As a last advantage, the pupils could not hear the feedback that is whispered by the coach (Scheeler & Lee, 2002). The computer does not expand our operation of immediate performance feedback. It simply permits us to deliver the procedure more efficiently and, possibly, more effectively (Froehle, 1984). 2.2.3.5
Content of immediate performance feedback
An immediate performance feedback message was coded as follows: 1) “classroom management” if the focus was on time, keeping space, or order on students; 2) “instruction” if the focus was on teaching curriculum; 3) “pedagogical” if the focus was on the relation between the pre-service teacher and a student; 4) “other” if the focus was not on one of the three abovementioned contents; and 5) “non-verbal” if the focus was on the non-verbal performance of the pre-service teacher. During the same lesson, different contents of feedback can be used. Of the 29 studies, the first three categories of content were used similarly: 11 studies used “classroom management” as content for feedback, 13 studies used “instruction,” and 10 studies used “pedagogical” relations between pre-service teacher and students as content for feedback. The category “other” consisted of examples of feedback on interview skills. In 10 studies, the non-verbal performances of pre-service teachers were addressed. Feedback is given on different aspects of performance. The aim is to train pre-service teachers in different more complex teaching skills, were different skills are interrelated. Pre-service teachers have a harder time keeping abreast of students’ work involvement patterns. Intermittent supplementary cues on the instruction process might aid in reminding pre-service teachers to continue monitoring the entire class group (van der Mars, 1988). This will be an especially helpful tool in increasing the quality of pedagogical relation and pre-service teachers’ classroom management (Rathel et al., 2008).
24
A review of technology-enabled immediate performance feedback 2.2.4
How was the immediate performance feedback given?
How immediate performance feedback was given depended on the place where the coach was sitting. When a coach was in the same classroom as a pre-service teacher, he could see and hear what was happening in the classroom. This had consequences for the used equipment. 2.2.4.1 Where was the immediate performance feedback given? The place where a coach sits has practical consequences for the observation of pre-service teachers. When a coach was in the same classroom as a pre-service teacher, he could see and hear the performance. If the coach was in an adjacent room with a one-way window, for example, he could see the performance of the pre-service teachers, but he could not hear what was happening in the classroom. If the coach was in another place (in another school or building), the coach had neither visibility nor sound of the performance of the pre-service teacher. For the 29 studies, 17 studies were conducted in the same classroom, 9 studies were in an adjacent room, and 3 studies were in a randomly assigned room. 2.2.4.2
Which technology was used to give immediate performance feedback?
When a coach was in the same room, a one-way communication set was more than sufficient. Because the coach could hear and see the performance, he didn’t have to receive sound (and visibility) of the pre-service teacher. When a coach was in an adjacent room, he could see the performance of the pre-service teacher, but he could not hear the classroom. A two-way communication set was then needed. If the coach was in a random room, he needed sound and visibility of the performance of a pre-service teacher. The coach could get sound of a two-way communication set and visibility of a camera placed in the classroom. The signals could be sent wirelessly to another room or building where the coach was sitting. Of the 29 studies, 17 studies used a one-way communication set, and 13 studies used a twoway communication set. 2.2.4.3 Where was immediate performance feedback given + used technology In the study of White and Gordon (2000), a remote location was supported by video monitoring (remote-controlled camera/videotaped and reviewed) and FM transmitters and receivers. The immediate performance feedback was sent directly to the pre-service teacher via a BIE. The feedback was sent through the Internet, which connected the coach’s computer to the pre-service teacher’s computer. The last five meters between the pre-service teacher and his or her computer were bridged with a Bluetooth connection. In the literature, this is called virtual coaching (Rock et al., 2009). Indeed, this is already being tried in universities with pre-service teachers conducting mini-lessons and/or student teaching at remote sites under the direction of faculty supervisors. Not only could university supervisors provide immediate feedback through the use of webcams, but they could do so with much greater frequency (Scheeler et al., 2004). However, participants noted several types of technologyrelated glitches encountered during the observation, forcing some to reschedule or redo a lesson (Rock et al., 2009). A more present disadvantage was that participants experienced heightened levels of anxiety around the observation and feedback process, resulting in reducing immediate performance feedback. (Rock et al., 2009).
25
Chapter 2
Alternatively, the sending could also be done through FM transmitters and receivers without the use of a remote location, equipped with two-way radio videoconferencing. A main advantage of this second option is that less equipment is required. Then, there are two options. The first is that the coach is in an adjacent room, most likely a room with a one-way mirror. The coach can see and (often) hear the pre-service teacher and sends immediate performance feedback to the teacher. Second, the coach is sitting in the back of the classroom and speaks softly into the microphone. It can be expected that in this last situation, the presence of the coach may serve as a confounding factor in pre-service teachers’ performance on the issue of observer reactivity (Rock et al., 2009). However, this last situation (in which the coach is in the same classroom) has four practical advantages. First, an emphasis is placed on the school as a learning environment where workplace learning is supported. It is used as initial and early professional learning of pre-service teachers. In this context, the pre-service teachers “take over the lessons” of a cooperating teacher, and these cooperating teachers are sitting in the back of the classroom. In the Netherlands, this is a common way of workplace-based learning for preservice teachers. Second, schools often are not equipped with adjacent rooms. This is especially not the case with one-way-mirrored windows. Third, the confounding factor of observer reactivity of a coach in an adjacent room or the same room influences the pre-service teacher equally. The pressure of the presence of a coach either in the classroom or in the adjacent room is, to our beliefs, almost the same, though (of course), this should be tested. The last advantage has to do with the number of studies conducted in the setting of the back of the classroom. There are more studies done in the actual classroom than a coach sitting in an adjacent room. We, therefore, propose that a coach is sitting in the back of a classroom sending immediate performance feedback though a one-way communication set. 2.2.4.4
Was there specific training necessary to give immediate performance feedback?
Different researchers stated that training time in the usage of BIE is required, and additional training is highly recommended (Giebelhaus & Cruz, 1992; Price et al., 2002). However, the training programs proposed were not consistent. This led to the question of whether or not there could be a general approach to immediate performance feedback. On the basis of their suggested training steps, we developed a protocol of necessary steps. A total of 19 studies explicitly described different training steps. By going through the articles and coding and summarizing the different steps they proposed (see Table 2-6), we derived an overall protocol of the training needed for immediate performance feedback with the use of BIE-technology. We found six general steps to follow with the second step being a training program for using BIE that consisted of five sub-steps. The different steps that a coach and a pre-service teacher should take to give immediate performance feedback are: 1) Taking care of two prerequisites (willingness to participate [both coach and pre-service teacher] and getting parental permission if students in the classroom are being filmed or interviewed for research purposes). 2) Implementing a training program for using BIE, consisting of five sub-steps: a) Technical instruction in using BIE;
26
A review of technology-enabled immediate performance feedback
3) 4)
5) 6)
b) Explanation of the targeted performance, “codes” (= in advance structured feedback messages) and what they mean; c) Explanation of the procedures and roles of the coach and the pre-service teacher; d) Modeling usage of BIE by the coach; and e) Practice time with the usage of BIE. Baseline measurement of the performance of the pre-service teacher. Intervention: the actual usage of BIE in the classroom: this preferable four times with each lesson where BIE is used ending in a reflection. In this reflection, the next targeted performance and codes should be discussed. Fading the usage of BIE (keeping BIE on but not giving immediate performance feedback and then removing BIE). Maintenance measurement of the performance of the pre-service teacher.
The criterion to include an additional step was taken from the study of Chambles and Hollon (1998). They suggested that when at least three well-designed studies with positive findings on any one intervention are evident in the literature, individuals may go forward with the assumption that they will be implementing an effective intervention.
2.3 A six-step approach to synchronous coaching In the literature search, we found common features in the usage of immediate performance feedback with BIE. Based on these features, we propose the consistent use of a method of giving immediate performance feedback using BIE. For this method, we propose the term synchronous coaching. The term synchronous coaching was first used in the study of Hooreman (2008) without explicit definition. Other terms of immediate performance feedback using BIE are used as portable radio communication system (Herold et al., 1971), covert audio coaching (Bennett et al., 2010), e-mentoring (Lord & Coninx, 2010) and transmitterassisted learning (Chow et al., 2001). No term is based on the implications of the literature. Table 2-6: Studies on protocols of using immediate performance feedback Article
Baseline measure
Trainings program
x x
x
x x
x
x
Baseline measure
Practice session
x
Successfully practicum
Modeling,
x x x x x
Role play
Explanation of the targeted behavior, prompts and what they mean
x X X x x
Explanation of the procedures and roles
Instruction in using BIE
x
Parental permission
Bennett, Brady, Scott, Dukes, Frain (2010) Bowles, Nelson (1976) Chow, Mak, Cheung, Louie (2010) Farrell, Chandler (2008) Fry, Hin (2006)
Willingness to participate
Reference
Administration
x x x
x
27
Chapter 2 Article Goodman, Duffy (2007) Goodman, Brady, Duffy, Scott, Pollard (2008) Gordon, Kogan (1975) Kahan (2000) Price, Martella, Marchand-Martella (2002) Rock, Gregg, Thead, Acker, Gable, Zigmond (2009) Scheeler, Lee (2002) Scheeler, McAfee, Ruhl, Lee (2006) Scheeler, Macluckie, Albright (2008) Scheeler, Bruno, Grubb (2009) Silverman, Kimmel (1972) Ward (1960) van der Mars (1988) White, Gordon (2000) Wimberger, Kogan (1974)
Administration
x x
x
x x
x
x
x x x
x x x
x x
x
x
x x x
x x
x x
x x
x x
x x
x x
x x x x x x x
x x x x x x x
x
x
x x
Baseline measure x x
x x
x x x
Trainings program x
x x x
x x
x x
x
x x
x x
x x
We have chosen for the term “coaching” because this method is also based on giving taskrelated feedback on targeted performance. Important to note is that pre-service teachers identified the performance that they wanted to change—not the researcher, not the observer, not the coach (Rock et al., 2009). We have chosen for the term “synchronous” because the feedback is given within three seconds after a targeted performance has occurred (Scheeler & Lee, 2002). This is different with performance feedback where immediacy is defined as feedback after teaching a lesson instead, such as a reflection conversation that occurs immediately after the lesson. For synchronous coaching, feedback is given while teaching a lesson. This article examines various usages of immediate performance feedback using BIE that arise from the literature and identifies common features of this usage. We have composed a six-step approach to synchronous coaching, consisting of the following steps: 1) Taking care of two prerequisites: a) Both coach and pre-service teacher must be willing to participate. b) Getting parental permission if pupils in the classroom are being filmed or interviewed for research purposes. 2) Implementing a training program for using a BIE device, which consists of five sub-steps: a) Explanation of the procedures and roles of coach and pre-service teacher. b) Determination of the target of the performance feedback—that is, which behavior should improve? As each behavior is associated with a set of possible feedback messages and the content of these messages are coded (= in advance structured feedback messages), the codes are elaborated and discussed. c) Technical instruction in using the BIE device. d) Modeling good teacher behavior while using the BIE device by coach. e) Practice time with using the BIE device. 3) Discussing performance of the pre-service teacher. 4) Intervention: the actual usage of the BIE device in the classroom is preferably four times (Farrell & Chandler, 2008; Fry & Hin, 2006; Kahan, 2002; O’Reilly et al., 1994; Price et al., 2002; Rock et al., 2009b), with each lesson ending in a reflection. In this reflection, the next targeted behavioral improvement and next codes should be discussed. 28
A review of technology-enabled immediate performance feedback 5) Fading the usage of BIE (keeping BIE in the ear, but not giving immediate performance feedback by coach and, at the end, not using the BIE device at all). 6) Evaluation of performance of pre-service teacher. This is in line with the study of Fry and Hin (2006), where the same common steps were found, such as taking care of prerequisites, a training program, an intervention, and maintenance measurements of performance. This is the only study, to our knowledge, that proposed a set of steps. Based on this, we propose a definition/description of synchronous coaching: In education, synchronous coaching is a method for giving immediate performance feedback with certain characteristics (specific, corrective, encouraging) by using BIE on targeted performance. These feedback messages are structured and discussed in advance. Most commonly, a cooperating teacher sits in the back of an actual classroom and sends the feedback to a pre-service teacher. A six-step approach with a training program for sending immediate performance feedback is preferable.
2.4 Conclusion and discussion Our aim was to contribute to a systematic way of measuring the effects of synchronous coaching. We proposed a definition of synchronous coaching and a six-step approach for synchronous coaching based on the implications of the literature. To our knowledge, only the study of Fry and Hin (2006) provided a defined step-by-step approach. By using a systemic way of approaching synchronous coaching, the measurement of the actual or potential rate of error can be obtained and cause and effect of changes on effectiveness can be established using independent testing methods. This definition and systematic approach finds its merit in the following research. Rather than impose a new construct definition on the field, this study addressed the need for construct clarification, using existing descriptions of common characteristics to create a snapshot of the synchronous coaching literature. Through a qualitative, literature-based approach, we were able to develop a definition of synchronous coaching. The definition sets boundaries, making it possible to compare cause-effect systems, outcomes, and results. By following a defined step approach, the effects can be measured in a systematic way, and the results of these effects could be compared and eventually combined. Until now, to measure the effects of synchronous coaching, different research methods have been used. The choice for a certain method depends on the common use of this method and the research approaches. In articles, presuppositions and restrictions are not always made explicit. Therefore, it is not our aim to state that these 29 articles, combined in this literature search, represent the body of knowledge of synchronous coaching. Nevertheless, these articles used several methods. Some articles used an experimental (for example, Bennett et al., 2010) or a semi-experimental (for example, Goodman et al., 2008) method; other researchers used a single case subject method (Wimberger & Kogan, 1974). All articles used a transparent method with detailed attention for the implications of the results. Coaching is an important process wherein a learning person gets feedback from a coach. Within the sector of education, synchronous coaching is a method for giving immediate, specific performance feedback to teachers by using a BIE in an actual classroom. In
29
Chapter 2 education, the coach is preferably a cooperating teacher, and the receiver of feedback is a preservice teacher. The targeted performance and feedback messages should be discussed in advance. The feedback messages should be specific, corrective, and (preferably) structured. We propose the use of a computer program for these specific feedback messages. The structured mode of delivering immediate performance feedback by use of a computer program has additional benefits for practice. A first one is that logs are available afterwards, which prove to be interesting for discussion on specific performance feedback. A second advantage relates to the presence of coaches in the classroom. O’Reilly and colleagues (1994) made a point about the requirement of the coach to change from unobtrusive observation to active involvement in the lesson. By using a computer program to deliver immediate performance feedback, a coach should not use his actual voice in the classroom, but by pressing a button, he can launch a message. The coach can be actively involved in the classroom without being obtrusive. A third advantage is that the use of a computer program enables a coach to whisper whatever and whenever he wants. By using a computer program, a coach has to select the keywords in advance. This computer program makes it possible to change from a coach-oriented direction to a student-oriented direction. By selecting the keywords, a coach will (probably) have more specific discussions in advance with the preservice teacher on the feedback that the pre-service teacher wants to get. This program makes it, therefore, impossible for a coach to give (simple) instructions and orders. A pre-service teacher can ask for more goal-oriented performance feedback on his/her specific behavior. The last advantage has to do with the coach. For the coach, although producing more cognitive load, a computer program can give a framework on specific, goal-oriented performance feedback of pre-service teachers. It is possible to develop a computer program that only includes keywords that represents evidence-based teacher behavior that has a positive effect on the learning of pupils. These keywords can act as a framework for discussing the performance of a pre-service teacher by a coach. Synchronous coaching can also be used outside the education sector. Research shows (for example, Kogan et al., 1972) that within the counseling sector, immediate performance feedback can be used in coaching in a one-on-one approach with his client. The proposed common features for synchronous coaching are formulated for the education sector; therefore, this sets a framework. In this framework, effects can be measured, respondents can be indicated, the setting can be defined, and a protocol and a training program can be formulated. Variations are possible, and every choice has its impact on the effects. Future research on synchronous coaching should include measuring effects on different levels. The four-level approach of Kirkpatrick and Kirkpatrick (2006) for measuring the effects of a training program can offer a framework. Kirkpatrick and Kirkpatrick stated that the effects of any training program can be measured on four levels: namely, the level of reaction (this is a measure of how participants feel about the various aspects of a training program), the level of Learning (this is a measure of the knowledge acquired, skills improved, or attitudes changed due to training), the level of behavior (this is a measure of the extent to which participants change their on-the-job behavior because of training), and the level of results (this is a measure of the final results that occur due to training, including higher productivity, less employee turnover, and improved quality). The fact is that most researchers aimed at making a contribution to a more effective coaching method. Giving the importance of support of pre-service teachers is a noble aim. There is
30
A review of technology-enabled immediate performance feedback potential for synchronous coaching to exist in the classroom, but there has to be permanent attention for refinements.
References Ashford, S.J. (1986). Feedback-seeking in individual adaptation: A resource perspective. Academy of Management Journal, 29(3), 465-487. Black, P., William, D. (1998). Assessment and classroom learning. Assessment in Education, 5(1), 7-75. Baum, D. (1976). An application of the “bug-in-the ear” communication system for training psychometrics. Counselor Education and Supervision, 15, 309-310. Bennett, K., Brady, M.P., Scott, J., Dukes, C., & Frain, M. (2010). The effects of covert audio coaching on the job performance of supported employees. Focus on Autism and Other Developmental Disabilities, 25(3), 173-185. Boe, E. E., Cook, L. H., & Sunderland, R. J. (2008). Teacher turnover: Examining exit attrition, teaching area transfer, and school migration. Exceptional Children, 75(1), 731. Bowles, E. P., & Nelson, R. O. (1976). Training teachers as mediators: Efficacy of a workshop versus the bug-in-ear technique. Journal of School Psychology, 14(1), 1525. Boylston, W., & Tuma, J. (1972). Training mental health professionals through the use of ‘a bug in the ear’. American Journal of Psychiatry, 129, 124-127. Brown, S. W. (2005). Emily and Rebecca: A tale of two teachers. Teaching and Teacher Education, 21(6), 637-648. Bullock, L. M., Gable, R. A., & Mohr, J. D. (2008). Technology-mediated instruction in distance education and teacher preparation in special education. Teacher Education and Special Education, 31(4), 225-300. Casey, J. A., Bloom, J. W., & Moan, E. R. (1994). Use of technology in counselor supervision. ERIC Digest, EDO-CG-94-25. Caldwell, B. J., Carter, E. M. A. (1993). The return of the mentor: Strategies for workplace learning. London: The Falmer Press. Chambless, D. L., & Hollon, S. (1998). Defining empirically supported therapies. Journal of Consulting and Clinical Psychology, 66, 7–18. Chandler, P., & Sweller, J. (1992). The split-attention effect as a factor in the design of instruction. British Journal of Educational Psychology, 62, 233-246. Chow, B. C., Mak, D.K.C., Cheung, S. Y., & Louie, L.H.T. (2001). Sharing of good practice: Use of the transmitter assisted learning. Hong Kong: Hong Kong Baptist University. Chow, B. C., Mak, D. K. C., Chung, S. Y., & Louie, L.H.T. (2010). Transmitter-assisted learning: New application in teaching and coaching sports skill. In: Studies on Teaching and Learning (pp. 93-98). ISBN 978-988-18105-5-7. Codding, R. S., Feinberg, A. B., Dunn, E. K., & Pace, G. M. (2005). The effects of immediate performance feedback on implementation of behavior support plans. Journal of Applied Behavior Analysis, 38, 205-219. Coleman, R. (1970). A conditioning technique applicable to elementary school classrooms. Journal of Applied Behavior Analysis, 3, 293-297. Coonen, H. (2005). De leraar in de kennissamenleving. [The teacher in a knowledge society]. Inaugural address. Heerlen. Open University. The Netherlands. Retrieved from http://www.ou.nl/docs/expertise/rdmc/oratie_huub_coonen.pdf.
31
Chapter 2 Coulter, G. A., & Grossen, B. (1997). The effectiveness of in-class instructive feedback versus after-class instructive feedback for teachers learning direct instruction teaching behaviors. Effective School Practices, 16, 21-35. Crasborn, F. J. A. J. (2010). The skilled mentor: Mentor teachers’ use and acquisition of supervisory skills. Unpublished doctoral thesis. Eindhoven, The Netherlands: Eindhoven School of Education. Crimmins, D. B., Bradlyn, A. S., St-Lawrence, J. S., & Kelly, J. A. (1984). A training technique for improving the parent-child interaction skills of an abusive neglectful mother. Child Abuse and Neglect, 8, 533-539. De Jonge, J. F. M., & de Muijnck, J. A. (2002). Waarom leraren de sector verlaten. Onderzoek naar de uitstroom in het primair en secundair onderwijs. Zoetermeer: EIM. Domoto, P. K., Weinstein, P., & Getz, T. (1979). A pilot study using remote broadcasting equipment to provide instruction in pedodontics. Journal of Dentistry Education, 43, 599-601. Douglas, A. J., & Rubin, S. (2011). Effectiveness of interactive computer-based instruction: a review of studies published between 1995 and 2007. Journal of Organizational Behavior Management, 31(1), 55-94. Duke, L., Karson, A., & Wheeler, J. (2006). Do mentoring and induction programs have greater benefits for teachers who lack pre-service training? Journal of Public and International Affairs, 17(2), 61-82. EC Directorate-General for Education and Culture. (2010). Developing coherent and systemwide induction programmes for beginning teachers: A handbook for policymakers.European Commission, SEC(2010)538 final. Farrell, A. C., & Chandler, D. (2008). Cooperating teachers’ impressions of the Whisper in my Ear (WIME) and traditional communication feedback methods for physical education pre service teachers. Education and Human Development, 2(1), 1-9. Feiman-Nemser, S. (2001). From preparation to practice: Designing a continuum to strengthen en sustain teaching. Teacher College Record, 103, 1013-1055. Field, A. (2009). Discovering statistics using SPSS (3rd ed.). Los Angeles: Sage. Froehle, T. (1984). Computer-assisted feedback in counseling supervision. Counselor Education and Supervision, 24, 168-175. Fry, J. M., & Hin, M. K. T. (2006). Peer coaching with interactive wireless technology between student teachers: satisfaction with role and communication. Interactive Learning Environments, 14(3), 193-204. Gallant, J. P., & Thyer, B. A. (1989). The “bug-in-the-ear” in clinical supervision: A review. The Clinical Supervisor, 7(2/3), 43-58. Giebelhaus, C. R., & Cruz, J. (1992). The third ear mechanical device: A supervision alternative. Journal of Early Childhood Teacher Education, 42(13), 8-12. Giebelhaus, C. R., & Cruz, J. (1994). The mechanical third ear device: An alternative to traditional student teaching supervision strategies. Journal of Teacher Education, 45, 365-373. Giebelhaus, C. R., & Cruz, J. (1995). Implementing the BIE intervention strategy with early field experience student teachers (Final Report No. ED393808). Giebelhaus, C. R. (1994). The bug-in-the-ear device: An alternative student teaching supervision strategy. Paper presented at the Association of Teacher Educators’ Annual Conference, Atlanta, GA. Giebelhaus, C. R., & Bowman, C. L. (2002). Teaching mentors: Is it worth the effort? Journal of Educational Research, 95(4), 246-254.
32
A review of technology-enabled immediate performance feedback Goodman, J. I. (2005). Increasing Learn Units by special education teachers: Supervision via bug-in-ear technology (Doctoral dissertation). Florida Atlantic University, Location. Dissertation Abstracts International-A 66/01, 144. Goodman, J. I., & Duffy, M. L. (2007). Using BUGS to increase student participation. Teaching Exceptional Children Plus, 3(4). Retrieved from: http://scholarship.bc.edu/education /tecplus/vol3/iss4/art3. Goodman, J. I., Brady, M. P., Duffy, M. L., Scott, J., & Pollard, N. E. (2008). The effects of ‘bug-in-ear’ supervision on special education teachers’ delivery of learn units. Focus on Autism and Other Developmental Disabilities, 23(4), 207-216. Gordon, D.A. (1975). Mobile, wireless ‘bug in the ear’ communication system for training and therapy. Behavior Therapy, 6(2), 130-132. Gordon, B. N., & Kogan, K. L. (1975). Mother instruction program: An analysis of intervention procedures. Family Process, 14, 205-221. Hattie, J. (2003). Teachers make a difference: What is the research evidence? University of Auckland, Australian Council for Educational Research. Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81-112. Hashweh, M. Z. (2003). Teacher accommodative change. Teaching and Teacher Education, 19, 421-434. Hennissen, P., Crasborn, F., Brouwer, N., Korthagen, F., & Bergen, T. (2008) Mapping Mentor teachers’ roles in mentoring dialogues, Educational Research Review 3, 168186. Herold, P., Ramirez, M., & Newkirk, J. (1971). A portable radio communication system for teacher education. Educational Technology, 11, 30-32. Hunt, D.D. (1980). Bug-in-the-ear technique for teaching interview skills. Journal of medical education, 55, 964-966. Ingersoll, R. (2001) Teacher Turnover, Teacher Shortages, and the Organization of Schools, University of Pennsylvania. American Educational Research Journal, 38(3), 499 – 534. Ingersoll, R. M. (2012). Beginning teacher induction: What the data tell us. Phi Delta Kappan, 93(8), 47 – 51 Kahan, D. (2002). The effects of a bug-in-the-ear device on intralesson communication between a student teacher and a cooperating teacher. Journal of Teaching in Physical Education, 22(1), 86-104. Katz, M. R., & Shatkin, L. (1983). Characteristics of computer assisted guidance. The Counseling Psychologist, 11, 15-31. Kersaint, G., Lewis, J., Potter, R., & Meisels, G. (2007). Why teachers leave: Factors that influence retention and resignation. Teaching and Teacher Education, 23(6), 775-794. Kirkpatrick, D., & Kirkpatrick, P. (2006). Evaluating training programs (3rd ed.). San Francisco, CA: Berrett-Koehler Publishers, Inc. Kluger, A. N., & DeNisi, A. (1996). The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 119(2), 254-284. Knight, P. (2002). Summative assessment in higher education: Practices in disarray. Studies in Higher Education, 27(3), 275-286. Kogan, K. L., Gordon, B. N., & Wimberger, H. C. (1972). Teaching mothers to alter interactions with their children. Childhood Education, 49, 107-110. Korner, I., & Brown, W. (1952). The mechanical third ear. Journal of Consulting Psychology, 16(1), 81-84.
33
Chapter 2 Krapfl, J. E., Bry, P., & Nawas, M. M. (1969). Uses of the bug-in the-ear in the modification of parents’ behavior. In R.D. Rubin & C.M. Franks (Eds.) Advances in behavior therapy (pp. 31-35). New York Academic Press. Kregten, A. V., & Moerkamp, T. (2004). Startende leraren in het buitenland. begeleiding van beginnende leraren in Canada, Engeland, Frankrijk, de Verenigde Staten en Zwitserland. [Beginning teachers abroad. Coaching of beginning teachers in Canada, England, France, United States and Switzerland]. Den Haag: SBO Locke, E. A., & Latham, G. P. (1984). Goal setting: A motivational technique that works. Englewood Cliffs, NJ: Prentice Hall. Lord, D., & Coninx, N. (2011). E-mentoring in vocational teacher education. In P. Isaias (Ed.), In Learning and instruction in Web 3.0. Advances in cognitive and educational psychology (pages of chapter/section). Springer Science. McClure, W. J., & Vriend, J. (1976). Training counselors using an absentee-cuing system. Canadian Counselor, 10, 120-126. Milich, D. R. (1975). Radio telemetry in clinical psychology and related areas. American Psychologist, 30, 419-425. Morris, J. (1974). An inexpensive, easy built, ‘bug-in-ear’/intercom system for training therapist in behavior modification techniques. Behavior Therapy, 5(6), 685-686. Neukrug, E. (1991). Computer-assisted live supervision in counselor skills training. Counselor Education and Supervision, 31, 132-138. Noell, G. H., Duhon, G. J., Gatti, S. L., & Connell, J. E. (2002). Consultation, follow-up, and behavior management intervention implementation in general education. School Psychology Review, 3, 217-234. O’Reilly, M. F., Renzaglia, A., Hutchins, M., Koterba-Bass, L., Clayton, M., Halle, J. W. et al. (1992). Teaching systematic instruction competencies to special education student teachers: An applied behavioral supervision model. Journal of the Association for Persons with Severe Handicaps, 17, 104-111. O’Reilly, M. F., Renzaglia, A., & Lee, S. (1994). An analysis of acquisition, generalization and maintenance of systematic instruction competencies by pre-service teachers using behavioral supervision techniques. Education and Training in Mental Retardation and Developmental Disabilities, 29, 22-33. Paas, F., Tuovinen, J., Tabbers, H., & Van Gerven, P. W. M. (2003). Cognitive load measurement as a means to advance cognitive load theory. Educational Psychologist, 38, 63-71. Peard, R., Hudson, P. (2006). Mentoring pre-service primary teachers in mathematics. In proceedings of EDU-COM 2006. Conference, engagement and empowerment: New opportunities for tertiary education in the 21st century (pp. 226-231). KhonKaen Thailand: Koen Khan University. Price, A. T., Martella, R. C., & Marchand-Martella, N. (2002). A comparison of immediate feedback delivered via an FM headset versus delayed feedback on the inappropriate verbalizations of a student with ADHD. Education and Treatment of Children, 25, 159-171. Ramaprasad, A. (1983). On the definition of feedback. Behavioral Science, 28(1), 4-13. Rathel, J. M., Drasgow, E., & Christle, C.C. (2008). Effects of supervisor performance feedback on increasing pre-service teachers positive communication behavior with students with emotional and behavioral disorders. Journal of Emotional and Behavioral Disorders, 16(2), 67-77. Rock, M. L., Gregg, M., Thead, B. K., Acker, S. E., Gable, R. A., & Zigmond, N. P. (2009). Can you hear me now? Evaluation of an online wireless technology to provide real-
34
A review of technology-enabled immediate performance feedback time feedback to special education teachers-in-training. Teacher Education and Special Education, 32(1), 64-82. Rock, M. L., Gregg, M., Howard, P. W., Ploessl, D. M., Maughn, S., Gable, R. A., & Zigmond, N. P. (2009). See me, hear me, coach me. Virtual bug-in-ear technology brings immediacy to professional development. National Staff Development Council, 30(3), 24-32. Ross, P. M., & Tronson, D. A. (2005). Providing quality feedback. Where to from here? Proceedings of Scholarly Inquiry into Science Teaching and Learning Symposium. Sydney, Australia. UniServe Science. Rushton, S. P. (2003). Two pre-service teachers’ growth in self-efficacy while teaching in an inner-city school. The Urban Review, 35(3), 167-189. Sadler, D. R. (1989). Formative assessment and the design of instructional systems. Instructional Science, 18, 119-144. Sadler, R. (1998). Formative assessment: Revisiting the territory. Assessment in Education, 5(1), 77-85. Scheeler, M. C., & Lee, D. L. (2002). Using technology to deliver immediate corrective feedback to preservice teachers. Journal of Behavioral Education, 11(4), 231-241. Scheeler, M. C., Ruhl, K. L., & McAfee, J. K. (2004). Providing performance feedback to teachers: A review. Teacher Education and Special Education, 27(3), 396-407. Scheeler, M. C., McAfee, J. K., Ruhl, K. L., & Lee, D. L. (2006). Effects of corrective feedback delivered via wireless technology on preservice teacher performance and student behavior. Teacher Education and Special Education, 29(1), 12-25. Scheeler, M. C., Macluckie, M., & Albright, K. (2008). Effects of immediate feedback delivered by peer tutors on the oral presentation skills of adolescents with learning disabilities. Remedial and Special Education, 11, 1- 10. Scheeler, M. C. (2008). Generalizing effective teaching skills: the missing link in teacher preparation. Journal of Behavioral Education, 17, 145-159. Scheeler, M. C., Bruno, K., & Grubb, E. (2009). Generalizing teaching techniques from university to K-12 classrooms: Teaching pre service to use what they learn. Journal of Behavioral Education, 18, 189-210. Shute, V. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153189. Silverman, S., & Kimmel, E. (1972). The influence of immediate feedback on the behavior of teachers in training. Final report. University of Florida, Tampa, and the National Center for Educational Research and Development. Regional Research Project No. 1D-069. Smith, H.D. (1984). Moment to moment counseling process feedback using a dual channel audiotape recording. Counselor Education and Supervision, 23, 346-349. Stumphauzer, J. S. (1970). A low-cost ‘bug-in-the-ear’ sound system for modification of therapist and parent behavior. Behavior Therapy, 67, 45-46. Sweller, J. (1990). Cognitive processes and instruction procedures. Australian Journal of Education, 34(2), 125-130. Sweller, J., Van Merriënboer, J. J. G., & Paas, F. (1998). Cognitive architecture and instructional design. Educational Psychology Review, 10(3), 251-296. Thomson, C. L. (1978). An experimental analysis of some procedures to teach priming and reinforcement skills to preschool teachers. Monographs of the Society for Research in Child Development, 43(4), 1-86. Van der Mars, H. (1988). The effects of audio-cueing on selected teaching behaviors of an experienced elementary physical education specialist. Journal of Teaching in Physical Education, 8(1), 64-72.
35
Chapter 2 Van Houten, R. (1980). Learning through feedback. New York: Human Sciences Press. Van Houten, R., Hill, S., & Parsons, M. (1975). An analysis of a performance feedback system: The effects of timing and feedback, public postings, and praise upon academic performance and peer interaction. Journal of Applied Behavior Analysis, 8, 449-457. Vries, S. de, Beijaard, D., & Buitink, J. (2008). Learning in the context of ‘co-creation of educational practices.’ In Ax, J. & P. Ponte (Eds.), The teaching profession in Dutch educational praxis (pp. 163-181). Publisher. Wade, Y. W. (2010). Increasing novice teacher support in 21st century classrooms: Induction and mentoring for beginning teachers through bug-in-ear technology (Doctoral dissertation). Orlando, Florida. University of Central Florida. Ward, C. (1960). An electronic aid for teaching interview techniques. Archives of General Psychiatry, 3(4), 357-358. Ward, C. (1962). Electronic preceptoring in teaching beginning psychotherapy. Journal of Medical Education, 37(12), 1128-1129. White, J., & Gordon, J. (2000). There’s a bug in my ear: Value adding through authentic interview experience. Contemporary Issues in Early Childhood, 2(1), 287-298. Wimberger, H. C., & Kogan, K. L. (1974). A direct approach to altering mother-child interaction in disturbed children. Arch. Gen. Psychology, 30, 636-639. Woullard, R., & Coats, L. T. (2004). The community college role in preparing future teachers: The impact of a mentoring program for pre-service teachers. Community College Journal of Research and Practice, 298, 609-624.
36
Chapter 3 The use of keywords for delivering immediate performance feedback2 Abstract: Literature shows that feedback which is specific, immediate and goal-oriented, is effective on (pre-service) teacher’s performance. Synchronous coaching gives this kind of feedback. Due to immediateness of feedback pre-service teachers can suffer from cognitive load. We propose a set of standardized keywords through which this performance feedback can be delivered; each keyword acts as a summary for the feedback message. The construction and the selection of the keywords is aimed to the reduction of message ambiguity, while at the same time a low level of cognitive load on the pre-service teacher must be maintained. An in vivo pilot-study with forty respondents (pre-service teachers and their coaches) supported our hypothesis that usage of such set of standardized keywords will mitigate the levels of ambiguity and cognitive load. These findings and other considerations for additional research using immediate performance are addressed. Keywords: performance feedback, BIE device, pre-service teachers, cognitive load, ambiguity
3.1 Introduction Many countries today are facing the issue of teacher shortages due to the demographic aging and drop-out of teachers early in their career (e.g. in the Netherlands Commissie Leraren, 2007). Teacher preparation and induction has, therefore, gained increased attention as a means of assisting teachers’ successful entry into the teaching profession (Giebelhaus & Bowman, 2002) and to reduce drop out. Studies have shown that successful entry into the profession reduces feelings of anxiety, isolation and powerlessness (Brown, 2005). However pre-service teachers reported a lack of effective support during their teacher preparation (Rushton, 2003). Consequently, effective teacher preparation of pre-service teachers has become a growing concern for teacher educators, preferably in the workplace, due to the increased emphasis placed on this. Rose and Church (1998) recommended practice with performance feedback as a necessary component of any training program that is implemented to change teacher behavior in the classroom. There is emerging evidence supporting the effectiveness of performance feedback (Codding et al., 2005; Noell et al., 2002) Therefore, an important teacher preparation technique is performance feedback. In this article we first elaborate on performance feedback: pre-service teachers should receive immediate (Scheeler & Lee, 2002), specific (Shute, 2008) and goal oriented feedback (Hattie & Timperley, 2007) on their performance within class rooms. We then introduce synchronous coaching as a method for delivering specific immediate feedback during class performance; in synchronous coaching we use a BIE (Bug In Ear) device which is actually an ear piece. 2
This chapter has been published as: Coninx, N. S., Kreijns, C. J., & Jochems, W. M. G. (2013). The use of keywords for delivering immediate performance feedback on teacher competence development. European Journal of Teacher Education, 36(2), 164–182.
37
Chapter 3 However, synchronous coaching is not without problems. We make clear that ad hoc whispering words or even whole sentences may introduce 1) interpretation problems regarding the content of the feedback message due to ambiguity and 2) cognitive load as a result that pre-service teachers have to divide their attention to both class and the interpretation of the feedback. Longer messages will therefore contribute to more cognitive (over) load. The solution is that we constructed a pre-defined set of keywords for decreasing the levels of ambiguity and cognitive load. We hypothesize that the use of a set of keywords will mitigate these problems as each keyword will act as a shortcut for the feedback message. The construction and selection of the keywords is our next topic. The keywords were constructed to provide specific feedback on the three main problems (Houtveen et al., 2006) pre-service teachers face (achieving effective classroom management, activating pupils towards their learning process, and catering for the individual needs of pupils). In order to test the hypothesis we executed an in vivo pilot-test in which two different modes were compared: 1) the traditional, asynchronous (ad hoc) way of whispering words or sentences (control group) and 2) the use of a set of keywords (experimental group).
3.2 Performance feedback Performance feedback is a systematic way to provide guidance to pre-service teachers while they acquire new skills (Noell et al., 2000; Sutherland et al., 2003). Performance feedback helped learners determine performance expectations, judged their level of understanding, and became aware of misconceptions. It also provided messages about the best approaches for correcting mistakes and improving performance (Kulhavy & Stock, 1989; Mory, 1992). To increase the effectiveness of performance feedback the feedback 1) should be specific (Shute, 2008), 2) goal oriented (Hattie & Timperley, 2007) and 3) should be immediate (Scheeler & Lee, 2002). Feedback is specific when it provides information about particular responses or behaviors (Shute, 2008). The feedback should be directed at the task (Black & Wiliam, 1998) and detailed (Ross & Tronson, 2005). Research findings demonstrated that when the pre-service feedback is focused, specific, clear and obtainable, the learning to teach is more effective (Peard & Hudson, 2006). The research findings also demonstrated that specific feedback resulted in positive changes in teacher behavior (Scheeler et al., 2004). According to Locke and Latham (1984) specific feedback was more effective than general feedback, primarily because it focused on the pre-service teachers’ attention, resulting that the feedback could be more directed. Specific feedback could also be made more manageable by narrowing the range of possible interpretations of the feedback reducing misunderstanding (Sweller, 1990). Possible meanings of a feedback message could cause ambiguity, resulting in higher levels of cognitive load. Ambiguity and cognitive load could lead to lower levels of learning (Kluger & DeNisi, 1996; Sweller et al., 1998) or even reduced motivation to respond to the feedback (Ashford, 1986). Goal-oriented feedback provides pre-service teachers with information about their progress towards a desired goal (or a set of goals) rather than providing feedback on discrete responses (i.e. responses to individual tasks) (Shute, 2008). Feedback should provide cues and information about teaching behavior that may lead to the development of more effective and efficient actions of pre-service teachers learning how to teach (Hattie & Timperley, 2007). Research has demonstrated that learners’ motivation and engagement depend on a close match between a learner’s goals and their expectations that these goals can be met (Fisher & 38
The use of keywords for delivering immediate performance feedback Ford, 1998). Therefore, communicating quality goal oriented feedback was a key element in enhancing learning. In this process, judgments are made about the match between evidence of achievements and standards (Knight, 2002). Quality feedback which promoted learning should include (non-evaluative) descriptions or evidence of the features of a teacher’s work, evaluative comments linked to criteria that indicate those features adding to or detracting from high quality, and setting targets for improvement (Sadler, 1989, 1998). Sadler (1989) highlighted that quality feedback can help learners to identify the learning gap between their current level of achievement and the desired (higher) level of attainment and support them to close this learning gap. Immediate feedback was defined as feedback delivered by the coach immediately after the target behavior occurred (Scheeler et al., 2008; Scheeler & Lee, 2002). Many field studies demonstrated the value of immediate feedback (Shute, 2008). Therefore, in teacher practice, feedback should be given preferably during class performance (immediate feedback) rather than after the performance (delayed feedback) to be most effective (Van Houten, 1980; Coulter & Grossen, 1997; O’Reilly et al., 1992; Scheeler et al., 2006). Immediate feedback has been used with pre-service teachers to effectively change faulty teaching behaviors (Rathel et al, 2008). Immediate feedback has a number of advantages. One advantage is that when immediate feedback was given coaches were able to give more feedback in less time (efficiency of learning), and coaches were able to model effective instruction techniques (O’Reilly et al., 1994). If coaches used immediate, specific performance feedback with pre-service teachers, this resulted in more opportunities to practice, faster (more efficient) acquisition of new behaviors by teachers, and more careful and efficient coaching sessions (Scheeler et al., 2006). Another advantage of using immediate feedback is that it provided learners with an opportunity to change target behaviors while practicing a skill, as opposed to their repeating errors until feedback is delivered, after the fact (Coulter & Grossen, 1997).
3.3 Synchronous coaching The underpinning pedagogical framework of synchronous coaching is a feedback mechanism (Codding et al., 2005; Farrell and Chandler, 2008; Bennett et al., 2010). A coach helps a preservice teacher to adopt evidence based actions of teaching by giving performance feedback. This feedback should include mastery criteria to ensure that the pre-service teacher can demonstrate effective teaching actions (goals). There are different ways to give immediate performance feedback. One way is that a coach can interrupt or even stop the lesson and give specific performance feedback. This method has obvious disadvantages such as a decrease of concentration and potentially a reduction of the self-esteem of the pre-service teacher. Furthermore, because of the disruption, there is value lesson time lost (Scheeler et al., 2006). A more preferred method is synchronous coaching. Synchronous coaching provides preservice teachers with immediate, on-the-spot feedback while these pre-service teachers are engaging in classroom instruction (Giebelhaus, 1994). Although researchers have used various forms of technology to provide immediate feedback, the BIE device is widely the most used device. Researchers had variously referred to this technology as a “Whisper-in-myear (WIME),” “wireless earphone,” a “mechanical third ear device,” or an “electronic audio-
39
Chapter 3 cueing system” (Rock et al., 2009). Most of us know BIE device as the tiny earpieces that people wear while they talk on their cell phones, instead of having to hold the phone to their ear. BIE device is referred to a small ear bud receiver that transmits verbal communication from a transmitter to a receiver. In general, BIE device has been used effectively in a number of disciplines to improve practices in the field (Franklin et al., 2007). BIE device has evolved from its initial use in psychotherapy (Korner & Brown, 1952) to many other professions including dentistry, special education, counseling, security, sports, and the general public (Goodman, 2005; Scheeler & Lee, 2002). Despite the knowledge base of synchronous coaching, it was not yet commonly used in the field of secondary education (Scheeler et al., 2006). When using BIE device in a teacher preparation situation, the coach delivered prompts and feedback to the pre-service wearing the ear piece to provide messages or cues for behavior. Synchronous coaching using BIE device provided immediate performance feedback during instruction that didn’t interfered with teaching, except to cue the pre-service teacher to use effective teaching behavior (Goodman et al., 2008). The pre-service was the only one to hear the prompts and can then decide what to say or do next (Goodman & Duffy, 2007). Research on this delivery of feedback that is not disruptive to the learning process of pre-service teachers is, therefore, urgent (Scheeler et al., 2004). 3.3.1
Reflections on synchronous coaching
Literature on synchronous coaching reported findings which are very promising. Pre-service teachers overwhelmingly gave the technology favorable reviews and stated that they could easily attend simultaneously to two sets of verbal stimuli (i.e. classroom students and a university supervisor) (Rock et al., 2009). Using this type of coaching allowed coaches to gave more cues on new to learn behaviors to pre-service teachers in less time, promoting efficiency of learning. In addition, such efficiency was likely to reduce frustration on the part of pre-service teachers who was attempting to acquire new behaviors but who otherwise must have waited for the reception of feedback on their performance (Scheeler et al., 2006). Although synchronous coaching is a promising technique, it was not without problems. Too frequent messages disrupt the pre-service teacher train of thoughts (Hunt, 1980). However the use of short, specific messages may have been helpful in preventing disruptions in the flow of instruction (Scheeler & Lee, 2002). The coach should restrict the length and frequency of his messages as a means of keeping nuisance and confusion factors at a minimum (Herold et al., 1971). Goodman et al. (2008) stated that besides restricting the amount of messages, the messages should also be predictable and should be used only in targeted lessons. Moreover, the appropriateness of the coach intervention can largely be averted if the coach and pre-service teacher agree in advance on the behavioral objectives they both wish to achieve (Herold et al., 1971). That way feedback message is more meaningful for the pre-service teacher (Chow, 2010) and it enables coaches to cue on performance outside the target area (Giebelhaus & Cruz, 1994). Coaches should not delay feedback messages for fear of unsubstantiated adverse instructional momentum effect. Coaches should however investigate ways to provide immediate performance feedback in the least intrusive manner. (Scheeler et al., 2006)
40
The use of keywords for delivering immediate performance feedback So, the feedback message has to be short to minimize the likelihood that the BIE device would be distracting (Scheeler et al., 2008) This was because at the moment of the actual communication two information sources competed for the attention of the pre-service teacher; the first being the actual classroom (visual and auditory information) and the second being the coach (using an BIE device: auditory information) and thus longer messages meant a longer staying in this state. In the cognitive load theory (Sweller 1990; Sweller et al., 1998) the division of attention could produce cognitive (over) load (Chandler & Sweller, 1992). Cognitive load is defined as “cognitive capacity that is actually allocated to accommodate the demands imposed by the task” (Paas et al., 2003, p. 64). In our study we want to construct keywords (i.e. short messages) that minimize cognitive load by reducing the number of words in the message. On the other hand, the message should also be clear. In various media theories, such as the media richness theory (Daft & Lengel, 1986) a clear message meant that the message should not display any ambiguity. The concept of ambiguity has a variety of different connotations, ranging from ambiguity in the sense of a double-interpretation in linguistics, to ambiguity in the context of lack of clarity (vagueness), and to ambiguity in situations where multiple seemingly contradictory - states or conditions could co-exist (Simpson & Weiner, 1999). Research literature has noted that ambiguity has been found to influence performance levels (DeRoma et al., 2003; Owen & Sweeney, 2002). Ambiguity was not easy to resolve but more information could alleviate the ambiguity. However, this could result in those lengthy messages that we just wanted to avoid. For effective synchronous coaching it is, therefore, crucial to find the optimum between a message as short as possible without suffering from ambiguity and therefore reducing the level of cognitive load of the pre-service teacher. 3.3.2
Keywords for synchronous coaching
In a meta-analysis of 83 studies of the reported problems of beginning teachers Veenman (1984) found that pre-service teachers have similar problems. From the list of problems, the three main problems almost every beginning teacher reported, are namely: achieving effective classroom management, activating pupils towards their learning process, and catering for the individual needs of pupils. Recent literature confirmed these findings (Stroot et al., 1998; Houtveen et al., 2006). For each of the three problems we searched in the literature for effective teaching actions for dealing with these problems. Effective teaching actions are evidence based teaching behaviors that help solve the problem. The effective teaching actions are found in the work of Marzano and colleagues (2001, 2009). For example: for the problem classroom management we found in the literature that ‘eye contact’ (Marzano et al., 2001) is an effective teaching action that helps to deal with classroom management problems. According to these effective teaching actions a pre-service is measured. These actions serve as goals for the performance of a preservice teacher. If a pre-service teacher feels that he/she do not use enough eye contact, he/she can get immediate feedback in an actual class room with synchronous coaching. These rationale is based on the following argumentation. Scheeler and colleagues (2004) stated that the responsibility for coaches is to specify more clearly what good teachers do and to give immediate performance feedback that increase those actions. Thomson and colleague’s (1978) do not suggest that teaching style is unimportant but rather that teaching 41
Chapter 3 approaches have been overbalanced towards style without sufficient attention to the measurability of style. So, each effective teaching action was defined in concrete (observable) class room behavior a teacher could undertake for increasing student achievement. Pronouncing such a concrete classroom behavior could be used as a feedback message on pre-service teachers’ performance and serve as a performance goal. Moreover, these feedback messages (who act as a summary for a certain effective classroom action) had to be specific because a pre-service teacher had to know what a coach meant by the message that was given on-the-spot. These feedback messages must be observable because a coach could only give immediate feedback on what he saw in the class room. And the feedback messages should be as short as possible in order to reduce cognitive load but without being ambiguous. These short, specific, goal oriented performance feedback messages are called ‘keywords’. Fifty eight keywords were selected. The column with heading ‘Feedback message Dutch’ were the exact messages that are given to the pre-service teachers.
3.4 Method The aim of this pilot study is to examine if the levels of cognitive load and ambiguity are higher in the control group (ad hoc whispering) than in the experimental group (structured whispering). Although the control group and the experimental are organized in the same way and follow the same protocol, they nevertheless differ in one independent variable: the use of the keywords. Coaches in the control group can make their own messages in advance on performance development regarding to the three problems of pre-service teachers (class room management, activating instruction and catering for the individual needs of pupils) where coaches in the experimental group use only the constructed, above mentioned, 58 keywords (Table 3-1). The control and the experimental group followed the same protocol (based on Farrell & Chandler, 2008) with a difference in step 2 and 3. The protocol consists of five steps: 1) Acquaint the pre-service teacher with wearing the BIE device. This is done in a lesson where no messages should be whispered. 2) Then, determine the competence to be developed and select appropriate keywords. Discuss the created keyword system (or create their own) to identify problems and solutions. For the experimental group, choose keywords from the constructed list (58 keywords of Table 3-1). For the control group, make your own keywords/sentence on the three problems of pre-service teachers. 3) In the same lesson a week later, give immediate specific performance feedback (selected keywords) by using the BIE device with standard keywords in the experimental condition and with own keywords in the control condition. 4) After the lesson, conduct a semi structured interview per couple (coach and pre-service teacher) 5) Fill in the questionnaire in and send it back to the researcher. This pilot study was carried out in a real life setting of a classroom with pupils. The implementation of the synchronous coaching mode is realized through the use of a BIE device. This device is a Sennheiser DW8000 wireless transmitter with earpiece. While a preservice teacher is teaching, a coach ‘whispered’ on-the-spot messages towards the pre-service teacher. The coach is sitting at the back of a class room for observing the behavior of a preservice teacher. In the Netherlands, this is a common way of coaching teachers. 42
The use of keywords for delivering immediate performance feedback Immediate performance feedback was defined as feedback delivered by the coach within three seconds after the target behavior occurred (Scheeler et al., 2008; Scheeler & Lee, 2002). This criterion was established in a separate study that looked at the latency between a student response and a teacher consequence (Scheeler & Lee, 2002). Table 3-1: Feedback messages (or keywords) for synchronous coaching
Non verbal behavior Feedback on Result
Class room management (handling order disturbing)
Marzano et al (2001,2009)
Nr.
Feedback message Dutch
Feedback message English
References
1 2 3 4 5 6
Basishouding Hoofd schudden Knikken Gebaren Verplaatsen Territorium
(d), (e), (f) (d), (e), (f) (d), (e), (f), (b), (d), (e), (f), (d), (e), (f) (d), (e),
7 8 9 10 11 12 13 14 15 16
Lichaamsorientatie Aanraken Oogcontact Fixeren Intonatie Stemvolume Pauze Tonen Lachen Afkeuren
Basic attitude (to stand, walk or sit) Shake head Nod Gestures Move Territory (getting into the territory of pupils) Body orientation Touch Eye contact Fixate Intonation Voice volume Pause Show (work of pupils) Laugh Disapprove
17 18
Aangeven wat correct is Aangeven wat correct is en waarom Geven van een suggestie Geven van een suggestie en waarom Aangeven van een goede oplossing Aangeven van een goede oplossing en waarom Aangeven van een standaard
Indicate what is correct Indicate what is correct is and to tell why Give a suggestion Give a suggestion and to tell why
(a), (b), (d), (e), (g), (f), (h) (b), (d), (e), (g), (f), (h)
Indicate a good solution
(b), (d), (e), (g), (f), (h)
Indicate a good solution and to tell why Indicate the standard
(a), (b), (d), (e), (g), (f), (h)
Probleem verhelderen Geven van hints Voordoen Begrip uitleggen Uitleg aanpak Leerling probleem preciezer omschrijven Leerling vragen wat al gedaan
Clarify the problem Give hints Show Explain the term Explain the structure Let a pupil exactly define the problem
(a), (b), (d), (e), (f), (h) (a), (d), (e), (f), (h) (a), (d), (e), (f), (h) (a), (d), (e), (f), (h) (a), (d), (e), (f), (h) (a), (b), (d), (f), (h)
Ask a pupil what he/she has already done Ask a pupil for his knowledge Stimulate a pupil his thinking process Ask if a pupil understands it
(b), (c), (d), (e), (f), (h) (a), (b), (c), (d), (e), (f), (h) (b), (c), (d), (e), (f), (h) (b), (c), (d), (e), (f), (h)
Instruction who Instruction what Instruction how Instruction time periode Instruction why Pronouncement Calculation mistake
(c), (d), (e), (g), (f) (c), (d), (e), (g), (f) (c), (d), (e), (g), (f) (c), (d), (e), (g), (f) (c), (d), (e), (g), (f) (d), (e) (d), (e)
19 20 21 22
Feedback on Process Instruction Conte nt
didactic instruction
23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40
Leerling vragen naar voorkennis Leerling denkproces stimuleren Leerling vragen of het begrepen is Instructie wie Instructie wat Instructie hoe Instructie tijdsduur Instructie waarom Uitspraak Rekenfout
(d), (e), (f), (d), (e), (f), (b), (c), (d), (e), (f) (d), (e) (d), (e), (f) (d), (e), (f), (b), (c), (d), (e), (f) (d), (e), (f), (d), (e), (f), (d), (e), (f), (h) (b), (d), (e), (f), (g), (h) (b), (d), (e), (f), (g),(h)
(a), (b), (d), (e), (f), (h)
43
Chapter 3 Marzano et al (2001,2009)
Nr.
Feedback message Dutch
Feedback message English
References
41 42 43
Writing mistake Next working form Select ad random pupils for answering Summarise Give every pupil attention Chalkboard neater Encouraging message for a pupil
(d), (e), (f) (b) (b) (a), f) (b), (f) (a) (b), (d), (e), (f), (g), (h)
48
Schrijffout Volgende werkvorm Kriskras leerlingen een beurt geven Samenvatten Alle leerlingen aandacht geven Bordschema ordelijker Leerling bemoedigende opmerking geven Leerling nonverbaal aanmoedigen
Non verbal encouraging for a pupil
(b), (d), (e), (f), (g), (h)
49 50 51 52
Leerling eruit sturen Leerling straf geven Leerling na de les laten nakomen Leerling verplaatsen
Throw out a pupil Punish a pupil Give detention to a pupil Move a pupil
(d), (e), (g), (h) (d), (e), (g), (h) (d), (e), (g), (h) (d), (e), (g), (h)
53 54
Leerling waarschuwen Leerling afkeuren met ikboodschap Leerling noemen en doorgaan les
Warn a pupil Disapprove a pupil with a I message
(d), (e), (g), (h) (d), (e), (g), (h)
Name a pupil and to continue the lesson Look at a pupil and to stop the lesson
(d), (e), (g), (h)
Encourage Disapprove
pedagogical action (relation pupil - teacher)
44 45 46 47
55 56
Leerling aankijken en stoppen van (d), (e), (g), (h) de les 57 Leerling aankijken en doorgaan Look at a pupil and to continue the (d), (e), (g), (h) les lesson 58 Leerling negeren Ignore a pupil (d), (e), (g), (h) (a) (Giebelhaus & Cruz, 1994), (b) (Goodman et al, 2008), (c) (Goodman & Duffy, 2007), (d) (Rathel et al, 2008), (e) (Rock et al., 2009), (f) (Scheeler et al., 2009), (g) (Hunt, 1980), (h) (Bowles & Nelson, 1976)
3.5 Participants Initially, thirty pre-service teachers were recruited from a teacher training institute for lower secondary education in southern Netherlands and ad random equally (fifteen each group) assigned to the control group or the experimental group. The pre-service teachers were assigned to a school coach and were asked to find a volunteering coach who liked to participate in this study. A pre-service teacher together with a volunteering coach formed a couple. During the training, from the thirty couples we lost ten couples (four in the experimental group and six in the control group) due to unavailability of the coach (2), sickness of the pre-service teacher (1), sickness of the coach (2), no suitable class room (1) and not functioning material (4). So, in this study 40 participants volunteered with nine couples in the control group and eleven couples in the experimental group. Twenty preservice teachers (13 males; 7 females) aged 17-34 years (M=22.4, SD=4.5) taught different subjects namely geography (4), history (3), Dutch (2), English (2), psychics (2), chemistry (3), and mathematics (4). Their pupils aged between 12 and 15 years. The number of pupils in a class room varied from 11 to 31 pupils. Twenty pre-service teachers found a volunteering coach. The coaches were experienced teachers, the pre- service teachers ‘took over their lessons’ while these experienced teachers were sitting in the back. In the Netherlands, this is a common way of workplace-based learning for pre-service teachers. The researchers have chosen for this approach because there
44
The use of keywords for delivering immediate performance feedback is some kind of trust between coach and pre-service teacher (Wong et al., 2004). The twenty coaches (4 males; 16 females) aged 28-54 years (M=38.9, SD=9.0), were working at one of eleven secondary schools. These schools are situated in southern and central Netherlands. The coaches had 3 to 20 years of experience (M=6.8, SD=5.8) in coaching pre-service teachers (Clutterbuck, 2004).
3.6 Instruments For each dependent variable (cognitive load and ambiguity) we used three different instruments: a questionnaire (instrument 1), a semi structured interview with coaches and preservice teachers (instrument 2) and the keywords logs (instrument 3). Logs (instrument 3) are differently produced for the control and the experimental group. Participants in the experimental group made use of the program CONPAS instead of actual whispering the keywords. CONPAS (Coaching of Newcomers Practical Assistant Software) is an in-house developed computer program that ensured that only the 58 pre-defined keywords were used. That is, pre-service teacher and coach determined which of the keywords were selected as the current subset for a certain competence development. At the very moment a coach observed less competent behavior of the pre-service teacher, she or he pressed one of the keywords in the current subset of keywords on a laptop to launch the message. Each keyword was a small audio tape of a female voice that pronounced the keyword. The pronouncement of a keyword could be activated by a mouse click (on a lap top). Because the keywords were pre-defined and audio taped in advance, the coach was enabled to ‘whisper’ other messages than the constructed set. CONPAS also made logs of the used keywords. The ad hoc mode (control group) was established by letting the coach directly whisper in the BIE device. The coach could whisper their own created messages on the three problems. We audio taped what the coach had whispered during the lesson and described this messages ad verbatim. The semi-structured interview (instrument 2) consisted of three topics: 1) how did the couple (pre-service teacher and coach) experienced a synchronous coaching session (which benefits, which problems) 2) did they experienced any problems regarding to sending and receiving messages (technical glitches, message to long, to many messages, the right moment for sending) 3) did they experienced interpretation problems (message clear, message useful). Cognitive load is measured with three different instruments using a questionnaire (instrument 1), an interview (instrument 2) and keywords logs (instrument 3). As for instrument 1, we used the one item 9-point cognitive load rating scale developed by Paas et al (1994). For instrument 2 we used a semi-structured interview in which coach and pre-service teacher were asked questions such as: were there too many messages, were messages too long? As for instrument 3 (keywords log) we used the average of words per given message. We counted the words and divided them by the number of messages. From the literature we know that the preferable number is three or four words per message (Scheeler et al., 2008). The rationale is that more than three or four words per message would produce cognitive overload. Ambiguity is also measured with three different instruments namely a questionnaire (instrument 1), a semi structured interview (instrument 2) and keywords logs (instrument 3). There was no existing questionnaire available that measures ambiguity. Therefore we 45
Chapter 3 developed one on the basis of the literature. Norton (1975) found eight characteristics for ambiguity by stating that ambiguity tolerance is : ‘the tendency to perceive or interpret information marked by 1. vague, incomplete or fragmented, 2. multiple, 3. probable, 4. unstructured, 5. uncertain, 6. inconsistent or contradictory, 7. contrary, 8. unclear meanings as actual or potential sources of psychological discomfort or threat.”. For each characteristic we formulated one item (e.g. ‘The given messages are unclear’ (with 1 = ‘I fully disagree’ to 5 = ‘I fully agree’). For instrument 2 we used a semi-structured interview in which coach and preservice teacher were asked questions like: Were the messages clear? Were the messages useful? For instrument 3 (keywords log) we used the content of the messages. The structured whispering mode can only use the keywords of the current subset representing one, two or all of the three main problems. In the ad hoc whispering mode we analysed to which problem (class room management, didactical instruction and catering for the individual needs of pupils) each message was referred. If there were other problems addressed (than the three main problems) the chance that a coach and a pre-service teacher had discussed this beforehand was small because of the given protocol (only address the three main problems). The rationale was that if there were other problems addressed without being discussed on beforehand, this can produce ambiguity. Interrater agreement data were collected on 95 of the 226 messages (42 %) of all verbatim messages. The first author coded each message as meeting the selection criteria or not the selection criterion (one of the three problems or not). A graduate research assistant provided agreement on each message using the same selection criteria. According to Bijou and colleagues (1968) inter rater agreement is the number of agreements divided by the sum of agreements (N=86) and disagreements (N=9). Inter rater agreement was .91 which is sufficient. For analysis of the semi structured interviews (instruments 2) we used the analysis of Westers and Peeters (2004). By a process of member check and peer debriefing we tried to guarantee the quality of the results. Two researchers interviewed the respondents. The data were analysed by two researchers separately. Each interview was written ad verbatim. Per interview one researcher searched towards codes (=clusters of answers). Exact agreements were considered as those in which the two researchers provided the same code. For the disagreements, the agreed consensuses for a code were used.
3.7 Results We measured two dependent variables: cognitive load and ambiguity. For cognitive load we used three different instruments. For instrument 1 an independent-samples t-test indicated a significant difference t(N)=20, p=.0.044 related to the cognitive load scale for the control group (M=4.33, SD=2.12), and the experimental group (M=2.45, SD=1.75). The magnitude of the difference in the means was large (Cohen’s d=1.03). Pre-service teachers experience significant less cognitive load in the experimental group (structured). Thus, the structured mode is more beneficial for the group of pre-service teachers in producing significant less cognitive load. A similar result found after analysing the data collected by the semi-structured interviews. The pre-service teachers of the experimental group experienced little to no cognitive load. The pre-service teachers of the control group reported more cognitive load. This was mainly the case when a pupil tries to explain something to the pre-service teacher at the same time as (long) messages of the coach were whispered. At that moment the information of two sources were competing for the attention of the pre-service teacher. Two pre-service teachers were disturbed by the long messages and had difficulties to continue their lessons.
46
The use of keywords for delivering immediate performance feedback “Sometimes I had to listen, really listen, and listen and listen again. While I was making notes on the chalkboard, I stood there and simply wait. I heard something, so I waited… and waited,… and waited until Anja (coach) stopped with whispering her long message. I had the strongest feeling that the pupils were looking at me with ‘You are waiting for what?’ This was a very strange feeling.” The experienced cognitive load was described in several ways. Two pre-service teachers mentioned that some messages produced ‘inner doubt’. “I have a good relationship with my coach and I take his criticism and tips very serious. But sometimes it was impossible to give attention to certain messages. Then, I found it annoying that I had to ignore her messages.” Cognitive load could also be caused because the message was’stored’ into the brain. “I didn’t react immediately to a message. I incorporated that message in my head and then tried it out at a certain moment something happened… and not immediately. That was more natural.” Also the use of ear bug can cause cognitive load. As the ear bug is not well placed into the ear, it asks for extra attention. “I had the feeling that my coach had laid down the transmitter for my ear piece so it felt if the communication was stopped. That gave me an uncertain feeling.” On analysis of the keywords (instrument 3) we found, once again, similar results. On average there are fewer words per message used in the experimental group (structured). This was significant different with the control group (ad hoc whispering). An independent-samples ttest was conducted to compare the average of used words per message. There was a significant difference in scores for the experimental group (M=1.91, SD=0.51), and the control group [M=10.07, SD=8.23; t(N)=20, p=0.04]. Taking the criterion of three or four keywords into account (Scheeler et al, 2008), we can conclude that the structured mode produce less cognitive load for pre-service teachers. Table 3-2: Average of used words per message Control group (ad hoc) experimental
Couple of coach and pre-service teacher
N messages used
N words used
Ad hoc Ad hoc Ad hoc Ad hoc Ad hoc Ad hoc Ad hoc Ad hoc Ad hoc Structured Structured Structured Structured Structured Structured Structured Structured Structured Structured Structured
Couple 1 Couple 2 Couple 3 Couple 4 Couple 5 Couple 6 Couple 7 Couple 8 Couple 9 Couple 1 Couple 2 Couple 3 Couple 4 Couple 5 Couple 6 Couple 7 Couple 8 Couple 9 Couple 10 Couple 11
11 40 21 20 18 59 8 22 27 5 2 7 4 9 4 3 8 9 3 5
57 187 101 501 270 350 35 474 116 12 4 14 9 19 9 4 9 16 7 10
N words used devided by N messages used 5.2 4.6 4.8 25 15 5.9 4.4 21.5 4.3 2.4 2 2 2.3 2.1 2.3 0.8 1.1 1.8 2.3 2
47
Chapter 3
For ambiguity we used also three different measurements: a questionnaire, an interview and an analysis of the keywords. For the questionnaire we found on one characteristic of Norton (unclear) a significant difference. An independent-samples t-test showed a significant difference related to perceived unclearness for the pre-service teachers of the experimental group (M=1.09, SD=0.30), and the control group [M=1.56, SD=0.52; t(N)= 20, p=.0.03]. The magnitude of the difference in the means was large (Cohen’s d=1.18).The higher scores of the control group represent a higher level of perceived unclearness. Reversed, for the experimental group, the levels of perceived unclearness are lower. Therefore there are indications to conclude that the structured mode produces less ambiguity for pre-service teachers. We found an equivalent result for ambiguity as a result of using instrument 2. In this semistructured interview it was asked if messages caused ambiguity. The pre-service teachers of the experimental group experienced almost no ambiguity. The pre-service service teachers of the control group experienced more ambiguity. On the question if the messages were understood, three out of nine pre-service teachers of the control group answered that they did not understand one or more messages. One pre-service teacher said: “There were several moments that I was thinking ‘What is she (coach) saying?’ I had totally no idea what was meant with certain messages. It had been more ideal when I received short commando such as ‘You should pay attention to Alex’ or ‘stop writing… the children aren’t paying any attention’ That would help much better.” Moreover, much effort must be done to understand the message entirely. On analysis of the content of the keywords (instrument 3) we found a similar result. Almost one message out of five (19%) didn’t address one of the three problems but gave information on other competence developments (42 of 226 messages). Therefore according to the protocol these messages were not discussed in advance between coach and pre-service teacher. This can have an effect on the on-the-spot understanding of a message of the pre-service teacher possibly resulting in more perceived ambiguity. Moreover, almost 74% (31 of the 42 messages) of the 42 messages not addressing one of the three problems, consisted of messages related to subject content competence. However, we know from Hooreman (2008) that subject content competence is not suitable to be synchronous coached because ”the pre-service teacher does not comprehend the whispering of the keywords of the behavior indicator of the subject content competence.”
3.8 Conclusion and discussion The results of this study suggest that the use of structured keywords for delivering immediate performance feedback is more beneficial than the ad hoc mode of delivering performance feedback on the three problems of pre-service teachers. This is in line with the literature about synchronous coaching. E.g. Farrell and Chandler (2008) stated that it was beneficial to create a ‘code system’ that allowed for minimal interaction. Beneficial is defined in terms of producing less cognitive load and ambiguity for pre-service teachers. For ambiguity, this seems logical because in the structured mode coach and pre-service teacher always have to select and discuss the keywords in advance that they are going to use. This could diminish the chance of ambiguity. In the ad hoc mode where the protocol determined that a coach only can use the keyword on the three problems, there is a considerable chance that the coach will use keyword addressing less easy to understand competences such as the subject content competence.
48
The use of keywords for delivering immediate performance feedback The reasoning related to cognitive load is slightly different. Walsh and colleagues (2009) stated that pre-service teachers have limited capacity such that they can attend to a finite amount of information simultaneously. The increased amount of cognitive processing required performing a task while receiving, interpreting and responding to immediate feedback may impose high extraneous cognitive load on pre-service teachers (Walsh et al., 2009). By structuring the messages and only using the selected keywords it is expected that cognitive load (i.e. extraneous load) can be reduced, resulting in lower reported levels of cognitive load. In addition significant smaller messages were used in the structured mode, resulting in lower levels of cognitive load. This is in line with previous research where was stated that the amount of cues should be restricted, predictable cues should be given and it should only be used for targeted lessons (Goodman et al., 2008). There are four limitations of this study. The first has to do with the number of respondents. Because of the nature of in vivo research the numbers of respondents are low. It would be beneficial to replicate this study with more teachers and to collect more data in different settings in order to support this intervention as a promising technique. A second limitation of this experiment is that there may be more factors contributing to the cognitive load of the structured mode that are not taken into account. Although our results are in line with the Cognitive Load Theory (Sweller, 1990), we concentrated only on the ‘extraneous load’ as a part of cognitive load. By structuring and reducing the keywords we have alleviated the chance that pre-service teachers are in a state of split attention. This is alleviating the extraneous load. But there are also two other parts of the cognitive load, such as the ‘intrinsic load’ (e.g. simultaneous activities in a class room) and the ‘germane load’ (e.g. working memory of the pre-service teacher). Recommendations for future research on decreasing cognitive load can include variations in class room size with fewer pupils and more experienced (e.g. four years) pre-service teachers. A third limitation pertaining to this experiment is that only three problems of pre-service teachers were addressed. It would be interesting in future research to determine what other keywords can be the effectiveness and social validity of each of these techniques in teacher preparation. A final limitation is that there wasn’t a scale to measure ambiguity. On the basis of the literature we composed a scale. Although the two other instruments give the same indication towards ambiguity, other scales of ambiguity should be used in future research concerning structured immediate performance feedback. Further research in the area of BIE device should include four domains. The first domain is that of investigating the maintenance of performance of pre-service teachers that was coached using BIE device. Most of the studies of immediate performance feedback address the acquisition of specific behavior. It is also desirable to investigate whether this behavior still exists after a certain period. The second domain includes research on the effects of delivering immediate performance feedback. Research on the effectiveness of a coaching program can be done on four levels according Kirkpatrick (2006) 1. level of reaction, 2. level of learning knowledge, skills and attitudes 3. level of behavior (change in behavior of pre-service teacher) 4. level of results (effects on pupils). The third domain contains research on the characteristics of a coach who is excellent in delivering immediate performance feedback. Also different educational settings can give a different picture for giving structured immediate performance feedback. The fourth domain of further research is about pairing with other technology such as for example videotaping. If professionals in the field of teacher education wish to improve the effectiveness of synchronous coaching, they should seek ways to implement the identified attributes in
49
Chapter 3 efficient and consistent ways (Scheeler et al., 2009). The challenges faced by teacher preparation programs are enormous. Structured immediate performance feedback could help pre-service teachers know what to teach, and how to teach is when they enter the school setting.
3.9 Acknowledgement We would like to thank Guido Lejeune for his help with the data collection.
References Ashford, S.J. (1986). Feedback-seeking in individual adaptation: A resource perspective. Academy of Management Journal, 29(3), 465-487. Black, P., William, D. (1998). Assessment and classroom learning. Assessment in Education, 5(1), 7-75. Bijou, S.W., Peterson, R.F., & Ault, M.H. (1968). A method to integrate descriptive and experimental field studies at the level of data and empirical concepts. Journal of Applied Behavior Analysis, 1(2), 175-191 Brown, S. W. (2005). Emily and Rebecca: A tale of two teachers. Teaching and Teacher Education, 21(6), 637-648. Chandler, P., & Sweller, J. (1992). The split-attention effect as a factor in the design of instruction. British Journal of Educational Psychology, 62, 233-246. Chow, B.C., Mak, D.K.C., Chung, S.Y., & Louie, L.H.T. (2010). Transmitter assisted learning: New application in teaching and coaching sports skill. In: Studies on teaching and learning (pp. 93-98).Hong Kong: Hong Kong Baptist University. ISBN 978-988-18105-5-7. Clutterbuck, D. (2004). Everyone needs a mentor: fostering talent at work. London, CIPD. Codding, R.S., Feinberg, A.B., Dunn, E.K., & Pace, G.M. (2005). The effects of immediate performance feedback on implementation of behavior support plans. Journal of Applied Behavior Analysis, 38, 205-219. Commissie Leraren [Dutch Teacher Commission] (2007), Leerkracht! ([Teacher!]). Den Haag: DeltaHage. Coninx, N.S, Kreijns, C.J., & Jochems, W.M.G. (2010). Perceived success of immediate coaching of student teachers with WIME (Whisper In My Ear) device. Paper presented at SITE conference, San Diego, United States of America. Coulter, G. A., & Grossen, B. (1997). The effectiveness of in-class instructive feedback versus after-class instructive feedback for teachers learning direct instruction teaching behaviors. Effective School Practices, 16, 21-35. Daft, R. L., & Lengel, R. H. (1986). Organizational information requirements: Media richness and structural design. Management Science, 1(5), 554-571. DeRoma, V.M, Kanetra, M.M., Kesler, M.L. (2003). The relationship between tolerance for ambiguity and need for course structure. Journal of Instructional Psychology, 30(2), 104-109. Farrell, A.C., Chandler, D. (2008). Cooperating teachers impression of the Wisper-In-My-Ear (WIME) and traditional communication feedback methods for Physical Education preservice teachers. Journal of Education and Human Development, 2(1), 1-9 Fisher, Ford (1998). Differential effects of learner effort and goal orientation and two learning outcomes. Personnel Psychology, 51(2), 397-420. 50
The use of keywords for delivering immediate performance feedback Franklin, T., Sexton, C., Lu, Y., & Ma, H. A. (2007). PDAs in teacher education: A case study examining mobile technology integration. Journal of Technology in Teacher Education, 15(1), 39-57. Fuller, F. (1969). Concerns of teachers: A developmental conceptualization. American Educational Research Journal, 6(2), 207-226. Giebelhaus, C. R., & Cruz, J. (1994). The mechanical third ear device: An alternative to traditional student teaching supervision strategies. Journal of Teacher Education, 45, 365-373. Giebelhaus, C. R. & Bowman, C. L. (2002). Teaching mentors: Is it worth the effort? The Journal of Educational Research, 95(4), 246-254. Goodman, J. I. (2005). Increasing Learn Units by special education teachers: Supervision via bug-in-ear technology. Unpublished doctoral thesis. [PLAATS, FL]: Atlantic University. Goodman, J.I., Duffy, M.L. (2007). Using BUGS to increase student participation. Teaching exceptional children plus, 3(4). Retrieved from: http://scholarship.bc.edu /education /tecplus/vol3/iss4/art3. Goodman, J.I., Brady, M.P., Duffy, M.L., Scott, J., Pollard, N.E. (2008). The effects of ‘bugin-ear’ supervision on special education teachers’ delivery of learn units. Focus on autism and other developmental disabilities, 23(4), 207-216. Hattie, J. & Timperley, H. (2007). The power of feedback. Review of educational research, 77(1), 81-112. Herold, P., Ramirez, M., & Newkirk, J. (1971). A portable radio communication system for teacher education. Educational Technology, 11, 30-32. Hooreman, R.W. (2008). Synchronous Coaching of Trainee Teachers, an experimental approach (ICO Dissertation Series No. 188) Unpublished doctoral thesis. Eindhoven, The Netherlands: Eindhoven School of Education. Houtveen, T., Versloot, B., & Groenen, I. (2006). De begeleiding van beginnende leraren. [Coaching of beginning teachers]. Utrecht, Sectiebestuur Onderwijsarbeidsmarkt (SBO). Retrieved from: http://www.onderwijsarbeidsmarkt.nl/fileadmin/user_upload /begeleiding_startende_leraren.pdf] Hunt, D.D. (1980). Bug-in-the-ear technique for teaching interview skills. Journal of medical education, 55, 964-966 Kirkpatrick, D. & Kirkpatrick, P. (2006) Evaluating training programs (3rd ed.). San Francisco, CA: Berrett-Koehler Publishers, Inc Kluger, A. N., & DeNisi, A. (1996). The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 119(2), 254-284. Knight, P. (2002). Summative assessment in higher education: Practices in disarray. Studies in Higher Education, 27(3), 275-286. Korner, I., & Brown, W. (1952). The mechanical third ear. Journal of Consulting Psychology, 16(1), 81-84. Kulhavy, R. W., & Stock, W. A. (1989). Feedback in written instruction: The place of response certitude. Educational Psychology Review, 1(4), 279-308. Locke, E. A., & Latham, G. P. (1984). Goal setting: A motivational technique that works. Englewood Cliffs, NJ: Prentice Hall. Marzano, R.J., Pickering, D.J., & Pollock, J.E. (2001). Class room instruction that works: research based strategies for increasing student achievement. Alexandria, VA: Association for Supervision and Curriculum Development. Marzano, R.J., Pickering, D.J., & Pollock, J.E. (2009). Wat werkt in de klas. [What Works in the class room]. Vlissingen, Bazalt.
51
Chapter 3 Mory, E. H. (1992). The use of informational feedback in instruction: Implications for future research. Educational Technology Research and Development, 40(3), 5-20. Noell, G. H.,Witt, J. C., LaFleur, L. H., Mortenson, B. P., Ranier, D. D., & LeVelle, J. (2000). Increasing intervention implementation in general education following consultation: A comparison of two follow-up strategies. Journal of Applied Behavior Analysis, 33, 271-284. Noell, G. H., Duhon, G. J., Gatti, S. L., & Connell, J. E. (2002). Consultation, follow-up, and behavior management intervention implementation in general education. School Psychology Review, 3, 217-234 Norton, R.W. (1975). Measurement of ambiguity tolerance. Journal of personality assessment, 39(6), 607-619. O’Reilly, M.F., Renzaglia, A., Hutchins, M., Koterba-Bass, L., Clayton, M., Halle, J. W. et al. (1992). Teaching systematic instruction competencies to special education student teachers: An applied behavioral supervision model. Journal of the Association for Persons With Severe Handicaps, 17, 104-111. O’Reilly, M. F., Renzaglia, A., & Lee, S. (1994). An analysis of acquisition, generalization and maintenance of systematic instruction competencies by pre-service teachers using behavioral supervision techniques. Education and Training in Mental Retardation and Developmental Disabilities, 29, 22-33. Paas, F., Van Merriënboer J.J.G., & Adam, J.J. (1994). Measurement of cognitive-load in instructional research. Perceptual and Motor Skills, 79, 419-430. Paas, F., Tuovinen, J., Tabbers, H., & Van Gerven, P.W.M. (2003). Cognitive load measurement as a means to advance cognitive load theory. Educational Psychologist, 38, 63-71. Peard, R., Hudson, P. (2006). Mentoring pre-service primary teachers in mathematics. In proceedings of EDU-COM 2006. Conference, engagement and empowerment: New opportunities for tertiary education in the 21st century (pp. 226-231). Koen Khan, Thailand: Koen Khan University. Rathel, J.M., Drasgow, E. & Christle, C.C. (2008). Effects of supervisor performance feedback on increasing pre-service teachers’ positive communication behavior with students with emotional and behavioral disorders. Journal of Emotional and behavioral disorders, 16(2), 67-77. Rock, M.L., Gregg, M., Thead, B.K., Acker, S.E., Gable, R.A., Zigmond, N.P. (2009). Can You Hear Me Now? Evaluation of an online wireless technology to provide real-time feedback to special education teachers-in-training. Teacher Education and Special Education, 32(1), 64-82. Rose, D.J., Church, R.J. (1998). Learning to teach: The acquisition and maintenance of teaching skills. Journal of Behavioral Education, 8, 5-35 Ross, P.M., Tronson, D.A. (2005). Providing quality feedback. Where to from here? In proceedings of Scholarly inquiry into science teaching and learning symposium (535). Sydney, Australia: UniServe Science. Rushton, S. P. (2003). Two pre-service teachers’ growth in self-efficacy while teaching in an inner-city school. The Urban Review, 35(3), 167-189. Sadler, D. R. (1989). Formative assessment and the design of instructional systems. Instructional Science, 18, 119-144. Sadler, R. (1998). Formative assessment: Revisiting the territory. Assessment in Education, 5 (1), 77-85. Scheeler, M. C., Bruno, K. & Grubb, E. (2009). Generalizing Teaching Techniques from university to K-12 classrooms: teaching pre-service to use what they learn. Journal of Behavioral Education, 18, 189-210.
52
The use of keywords for delivering immediate performance feedback Scheeler, M.C., Lee, D.L. (2002). Using technology to deliver immediate corrective feedback to pre-service teachers. Journal of behavioral education, 11(4), 231-241. Scheeler, M.C., McAfee, J.K., Ruhl, K.L., Lee, D.L. (2006). Effects of corrective feedback delivered via wireless technology on pre-service teacher performance and student behavior. Teacher education and special education, 29(1), 12-25. Scheeler, M.C., Ruhl, K.L., McAfee, J.K. (2004). Providing performance feedback to teachers: a review. Teacher education and special education, 27(3), 396-407. Scheeler, M.C., Macluckie, M., Albright, K. (2008). Effects of immediate feedback delivered by peer tutors on the oral presentation skills of adolescents with learning disabilities. Remedial and special education, 11, 1-10. Shute, V. (2008). Focus on formative feedback. Review of educational research, 78(1), 153189. Simpson, J. A. and E. S. Weiner (1999). The Oxford English Dictionary. Oxford, Oxford University Press: CD-ROM. Sutherland, K. S., Alder, N., & Gunter, P. L. (2003). The effects of varying rates of opportunities to respond to academic requests on the classroom behavior of students with EBD. Journal of Emotional and Behavioral Disorders, 11, 239-248. Sweller, J. (1990). Cognitive processes and instruction procedures. Australian Journal of Education, 34(2), 125-130. Sweller, J., Van Merriënboer, J. J. G., & Paas, F. (1998). Cognitive architecture and instructional design. Educational Psychology Review, 10(3), 251-296. Van Houten, R. (1980). Learning through feedback. New York: Human Sciences Press. Veenman, S. (1984). Perceived problems of beginning teachers. Review of Educational Research, 54(2), 143-178. Walsh, C.M., Ling, S.C., Wang, C.S., Carnahan, H. (2009). Concurrent feedback versus terminal feedback: it may be better to wait. Academic Medicine, 84(10), 54-57. Westers, F., Peters, V. (2004). Kwalitatieve analyse. Uitganspunten en procedures. [Qualitative analyses. Main points and procedures]. Bussum: Uitgeverij Coutinho. Wong, H.K., Britton, T., Gasner, T. (2004). What the world can teach us about improving new teacher induction. Phi Delta Kappan, 86(5), 379-384.
53
Chapter 3
54
Chapter 4 Perceived satisfaction of pre-service teachers3 Abstract: Review of the current literature on feedback revealed that when it is specific, immediate, and goal-oriented, the feedback will be effective on (pre-service) teachers’ performance. Synchronous coaching by using a bug-in-ear (BIE) device provides this kind of feedback, and we composed a systematic approach for using synchronous coaching. The aim of the present study was to measure the effects of synchronous coaching on the first level of effectiveness of Kirkpatrick, namely perceived satisfaction. To this end, a literature search was conducted on perceived satisfaction of synchronous coaching. An in vivo study with 80 respondents (pre-service teachers and their coaches) supported the literature findings that according to pre-service teachers, synchronous coaching is more supportive, is providing more useful feedback, and is more encouraging than traditional, asynchronous ways of coaching. These findings and other considerations for additional research using synchronous coaching are addressed. Keywords: synchronous coaching, immediate specific performance feedback, bug-in-ear device, WIME device, immediate feedback
4.1 Introduction Pre-service teachers seem ill-prepared for the teaching job. In education, beginning and novice teachers are expected to assume the same roles and responsibilities as an experienced teacher on the very first day of employment, and on-the-job training is limited to the student teaching practicum. Current pre-service teachers are expected to bridge the gap between theory and practice during student teaching, in which the implementation of learned instructional strategies is applied in the actual classroom (Wade, 2010). A number of studies showed that pre-service teachers may experience feelings of anxiety, isolation, and lack of support (Brown, 2005; Rushton, 2003; Smith & Thompson, 2005; Woullard & Coats, 2004). An increased number of pre-service teachers described feeling unprepared for generaleducation classrooms and even less prepared for diverse learners (Brownell, Hirsch, & Seo, 2004). The teaching profession has alarmingly high rates of dropouts during the induction phase due to the lack of support during this reality-shock experience. Teacher preparation programs are challenged to improve pre-service teachers’ preparation to decrease the gap between theory and practice (Billingsley, Carlson, & Klein, 2004; Boe, Cook, & Sunderland, 2008; Kersaint, Lewis, Potter & Meisels, 2007). Hasweh (2003) called for a more systematic and empirically validated research that is directed towards the support and optimization of teachers’ professional development. The current study aimed to contribute to teacher-education programs by: viewing synchronous coaching as one element of a more comprehensive pre-service-teacher-education program; complementing the
3
This chapter is submitted for publication as an article titled “Perceived satisfaction of pre-service teachers”.
55
Chapter 4 behavior of pre-service teachers with competences they will use in their professional lives; and supporting pre-service teachers in their work towards becoming professional teachers. High rates of turnover of professional teachers contribute significantly to the attrition of preservice teachers in education (Darling-Hammond, 2000; Smith & Ingersoll, 2004). Facing teacher shortages, teacher-preparation programs have gained increased attention as a means of assisting pre-service teachers’ successful entry into the teaching profession (Giebelhaus & Bowman, 2002). Researchers have focused on efficient and effective means for preparing preservice teachers to reach the higher standards imposed by a changing society and to stay in the teaching profession (Darling-Hammond & Brandsford, 2005). The goal of this research is to determine whether synchronous coaching will support preservice teachers during their teacher training phase. Preparing pre-service teachers for diverse populations of pupils is an important task that must be addressed systematically. Current teacher-education literature addresses the need for standards and accountability and stresses the importance of well-researched teacher-preparation methods (Goodman & Duffy, 2007). Scheeler, Ruhl, and McAfee (2004) provided evidence that teacher instruction improved when a coach gave specific, immediate, and goal-oriented performance feedback to pre-service teachers. Teachers—like other learners—acquire and maintain new behaviors best when they have multiple opportunities to practice and receive performance feedback that is specific, immediate, and goal-oriented, i.e., synchronous coaching. The hypothesis that was tested, and whose results are described herein, is: Pre-service teachers in the control group are less satisfied about synchronous coaching and received less feedback than pre-service teachers in the experimental group. Satisfaction is determined for the five functions of coaching, of which supportiveness and giving encouragement are the two functions that can decrease teacher attrition during the teacher training phase. The hypothesis is tested in a specific playing field. The synchronous coaching needs to be done with the use of BIE device and keywords as described in Coninx, Kreijns, and Jochems (2013). First, a theoretical framework as described in the second section is constructed. Then a literature research is done to collect the existing knowledge on supporting or disproving the hypothesis. This existing knowledge is evaluated when the specific boundaries in this research are used. In the last paragraph, the conclusions are drawn and are discussed for future research possibilities.
4.2 Theoretical framework The research contains the effect of synchronous coaching, with the use of bug-in-ear (BIE) technology, on the first level of Kirkpatrick’s four-level evaluation model (Kirkpatrick, 2002). In this section, the different components are described. 4.2.1
Synchronous coaching
Coaches can provide specific goal-oriented feedback in two ways, i.e., delayed or immediate (Scheeler, McAfee, Ruhl, Lee, 2006; O’Reilly, Renzaglia & Lee, 1994). In delayed feedback, coaches take notes while observing a lesson and share these notes with the pre-service teacher
56
Perceived satisfaction of pre-service teachers upon lesson completion, perhaps even later. However, post-observation conferences are most effective when they occur soon after the observation (Swafford, 1998). The disadvantage of delayed feedback is that the pre-service teacher may practice less effective methods, offering students less effective instruction, thus causing them to possibly acquire ineffective or inaccurate skills. In addition, delayed feedback makes it difficult for both coach and preservice teacher to reconstruct personal examples of what was occurring in the targeted teaching interchange, thus reducing the feedback’s effectiveness (Scheeler et al., 2006). Immediate feedback does not have this disadvantage. In immediate feedback, coaches observe a problem in pre-service teacher behavior (or lack thereof), instantly stop the instructional activity, and correct the pre-service teacher’s behavior directly during the observation (Scheeler et al., 2006). Immediate performance feedback thus allows coaches to provide pre-service teachers with immediate, on-the-spot feedback while these pre-service teachers are engaging in classroom instruction (Giebelhaus & Cruz, 1992, 1994). Therefore, immediate feedback, although interrupting and disturbing classroom instruction, is usually preferred. The review suggests that although there has been research involving the support of synchronous coaching for pre-service teachers, there is only limited research regarding the link between a systematic approach of synchronous coaching and measuring the effectiveness. 4.2.2
Bug-in ear technology
The interruption and disturbance of immediate feedback can be avoided by the use of technology, as this enables unobtrusive immediate feedback while pre-service teachers receive real-time, on-the-spot feedback during classroom experiences. Although researchers have used various forms of technology to provide immediate feedback, the bug-in-ear is the most used device. Researchers have variously referred to this technology as a “whisper-inmy-ear” (WIME), “wireless earphone,” “mechanical third ear device,” or “electronic audiocueing system” (Rock et al., 2009a). Most of us know BIE device as the tiny earpieces that people wear while they talk on their cell phones, instead of having to hold the phone to their ear. BIE device refers to a small earbud device that receives verbal communication sent by a transmitter. When using BIE in a teaching or coaching situation, the coach delivers prompts and feedback to the person wearing the earbud to provide directives or cues for behavior. Using BIE in the classroom provides immediate, specific feedback during instruction that does not interfere with teaching, except to cue the teacher to use effective instructional methods (Goodman, Brady, Duffy, Scott, & Pollard, 2008). This allows for a more natural environment for the student. The student is the only one to hear the prompts and can then decide what to say next (Goodman & Duffy, 2007). In a previous extensive literature search by Coninx, Kreijns, and Jochems (2012), a systematic approach of synchronous coaching was developed on the basis of 29 articles. This has led to the definition: synchronous coaching is a method for giving immediate performance feedback with certain characteristics (specific, corrective, encouraging) by using a BIE device on targeted performance. Before the lessons start, the target behavior of the performance feedback is determined and, therefore, the associated set of possible feedback messages is set. As the contents of these messages are coded (i.e., short and unambiguous determined and constructed feedback messages), their meaning is elaborated and discussed in advance. In the 57
Chapter 4 most common setting, a coach or a colleague sits in the back of an actual classroom and, when appropriate, sends the feedback messages to a pre-service teacher. 4.2.3
First level of reaction
There is a need to investigate the potential effect of synchronous coaching to pre-service teachers. The best-known evaluation methodology of training programs is Donald Kirkpatrick’s four-level evaluation model (Kirkpatrick, 2002). For the purpose of this study, we focus on Kirkpatrick’s first level of reaction, which in his own words is: …a measure of how participants feel about the various aspects of a training program. Reaction is basically a measure for customer satisfaction. It’s important because school management often makes decisions about a training program based on participants’ comments. Another reason for measuring is to ensure that participants are motivated and interested in learning. If they don’t like the program, there is little chance that they will put in the effort. (p. 56) It is for this particular reason that we first have investigated perceived satisfaction before we proceed to the other effects, such as change in the teacher’s behavior.
4.3 Literature research A systematic literature search on synchronous coaching regarding perceived satisfaction (e.g., the first level of reaction), based on Kirkpatrick (2002), was conducted. 4.3.1
The research method
The ERIC search used (a combination of) the following terms: “immediate feedback,” “performance feedback,” “bug-in-ear,” “wireless communication,” and “teachers.” Articles were selected if they included one or more of these terms. Articles including the above search terms in combinations in their title or abstract and focusing on immediate feedback were included in the overview. The “snowball” method was applied by examining the references of the articles, which led to other relevant articles. The raw yield was 61 articles. These were further examined on additional criteria, including the clarity of the described method section and the use of an actual BIE device. Studies in which the researchers used immediate email after the lessons (Rathel, Drasgow, & Christle, 2008) or researchers who used keyword entry placed on a monitor (Neukrug, 1991) were excluded because of the lack of the use of a BIE device. Thus, excluded sources were either off-topic or poorly designed. On the basis of these criteria, the yield was 29 articles. These 29 articles4 were analyzed as to what extent they
4
The literature search revealed that most of the studies (23 of the 29 studies) were carried out in the education sector with 49 percent in primary education, 26 percent in special education and 17 percent in secondary school education. Also, the majority of the studies (18 of the 29 studies) reported qualitative data. Twelve of the 29 studies made use of statistical analyses and therefore have been termed quantitative data. Of the 29 studies, only eight studies used an quantitative comparison between a control group and an experimental group. The numbers of respondents are relatively low, going from a single subject to 34 respondents. White and Gordon (2000) is an exception with a study of 60 participants.
58
Perceived satisfaction of pre-service teachers answered the question: “To what extent did these articles report the effects on the level of ‘reaction’ (e.g. satisfaction) as defined by Kirkpatrick?” The literature search on the level of reaction (Kirkpatrick, 2002) is well investigated. Sixteen studies of the 29 studies reported effects on the level of Reaction. 4.3.2
Results
In general, pre-service teachers were satisfied with synchronous coaching. In a study of Fry and Hin (2006), a questionnaire was administered by 21 pre-service teachers to examine the level of satisfaction with synchronous coaching. Although the pre-service teachers indicated that the equipment was sometimes inconvenient and disruptive during their lessons, they generally said they were satisfied with this way of coaching. However, Ward (1960) was somewhat amazed by how readily the BIE device has been adopted by teachers. He reported that the BIE device is not strange and that students had minimal difficulty receiving comments and adapting them to their own style. In 1962, Ward reported that students’ reactions to synchronous coaching were systematically surveyed, and he found that 80 percent of the students reacted favorably to synchronous coaching. When there were complaints, they most frequently had to do with technical problems, principally poorly adjusted volume, battery failure or loose-fitting earpieces. The coaches were somewhat slower than the students to accept the BIE device; not infrequently, it was at the students’ insistence that the reluctant teacher first tried this new method. Seventeen of the 20 teachers employed it regularly. Similar results were found in the study of Domoto, Weinstein, and Getz (1979). The questionnaire indicated that although the experimental and the control groups had equal exposure to the treatment, the synchronous coaching group found their experience to be more positive. Especially, the synchronous coaching group reported that they would strongly recommend the experience to other students (80 percent) while only 33 percent of the control group made similar recommendations. If we look more closely at what is meant by satisfaction, we find the terms like “supportiveness,” “usefulness” (i.e., of feedback), and “helpful.” In a study of Rock and colleagues (2009a), 15 pre-service were asked to keep written reflections. Qualitative analysis of this reflection reports revealed that the pre-service teachers were very satisfied with immediate feedback. Moreover, 53 percent of pre-service teachers found this coaching method supportive, 73 percent useful and 6 percent positive. Not only were the pre-service teachers able to respond directly (i.e., change their behavior) to the feedback without interruption in their classroom, 73 percent of them reported that immediate feedback led to better understanding of their practices in the classroom. Rock et al. (2009a) reported: When a coach watches you teach in person, there is no immediate feedback, which a teacher really needs. With the BIE device, there is immediate feedback even while teaching. Throughout all of my lessons, Dr. [X] was telling me how I was doing and what I needed to do, which gave me confidence in my ability to get the job done. In my past experience, a professor would come out to the classroom and observe a lesson and then provide feedback after the lesson. While this was helpful when planning future lessons, it did nothing for the lesson I had completed. (p. 78)
59
Chapter 4 The usability was investigated in a study by Domoto, Weinstein, and Getz (1979). Ten of 20 students were coached synchronously. The other ten students discussed their own video recordings immediately after the lesson. The results showed that synchronous coaching is a useful method. Initially the students feared that the coach’s feedback messages would be distracting and would affect them adversely. However, probably due to the encouraging tone of the feedback messages, the student teachers had their perceptions changed. The study by Chow, Mak, Cheung, and Louie (2001) confirmed the usefulness of synchronous coaching in micro-teaching situations. Hunt (1980) did an interview with 34 students, and the majority of comments from students were positive. Students’ reactions to the technique were similar, with 76 percent being anxious prior to the experiment but not significantly bothered during the experiment. Three students (8 percent) reported being bothered throughout the experiment; especially, they reported being distracted by the comments, and a poor microphone connection of resulted in one student hearing an unfortunate degree of static. The majority of the comments were positive. The students generally appreciated the feedback and the assistance. After completing the study, all teachers were asked two questions to assess the social acceptability of the intervention in the study of Scheeler and Lee (2002). All teachers responded that receiving immediate feedback was helpful and using the BIE device was not distracting to them or their students. In the study of Chow and colleagues (2001) twelve preservice teachers were interviewed, and they all commented that synchronous coaching was helpful in (micro-) teaching situations.
4.4 Research method We conducted an in vivo study with an experimental group (synchronous coaching) and control group (asynchronous coaching) to verify the findings of the literature search. 4.4.1
Participants
Forty pre-service teachers were recruited from a teacher-training institute for lower secondary education in the southern Netherlands and at random equally (20 each group) assigned to the control group or the experimental group. The pre-service teachers were asked to find a volunteering coach. All 40 coaches had previously served as a coach. All coaches received positive recommendations from former pre-service teachers as being motivated and highly skilled professionals. Coaches were willing and received permission from necessary supervisors to coach pre-service teachers for the spring semester. The characteristics of being skilled and having previously been utilized as a coach qualified them for the study (Farrell & Chandler, 2008). A pre-service teacher together with a volunteering coach formed a pair. However, from the 40 pairs, 10 pairs dropped out (five in each group) due to unavailability of the coach (2), sickness of the pre-service teacher (1), sickness of the coach (2), no suitable classroom (1), and non-functioning technology (4). So, 60 participants volunteered, with 15 pairs in the control group and 15 pairs in the experimental group. Thirty pre-service teachers (17 males, 13 females) aged 17 to 34 years (M=21.7, SD=3.8) taught different subjects, namely geography (10), history (5), Dutch (3), English (3), physics (2), chemistry (3), and
60
Perceived satisfaction of pre-service teachers mathematics (4). Their pupils were between 12 and 15 years old. The number of pupils in a classroom varied from 11 to 31. The 30 coaches were all experienced teachers. The pre-service teachers took over the coaches' classrooms while the coaches were sitting in the back. In the Netherlands, this is a common means of workplace-based learning for pre-service teachers. The researchers have chosen this approach because it establishes a certain comfort level between the coach and pre-service teacher (Wong et al., 2004). The 30 coaches (8 males; 22 females) aged 28-54 years (M=41.4, SD=8.7), were working at 11 secondary schools, situated in the southern and central parts of the Netherlands. The coaches had 3 to 20 years (M=8.5, SD=4.7) of experience in coaching pre-service teachers (Clutterbuck, 2004). 4.4.2
Instrument
There are three instruments used in this research: the six-step approach to synchronous coaching, the use of BIE device, and the questionnaire. Six-step approach: A protocol for using synchronous coaching in an actual classroom is determined (Coninx et al., 2013). This systematic approach is a six-step approach with a training program for using immediate performance feedback. These six steps are summarized as follows: 1) Taking care of two prerequisites: a) Both coach and pre-service teacher must be willing to participate. b) Getting parental permission if students in the classroom are being filmed or interviewed for research purposes. 2) Implementing a training program for using a BIE device consisting of five sub-steps: a) Explanation of the procedures and roles of coach and pre-service teacher. b) Determining the target of the performance feedback, i.e., which behavior should improve? As each behavior is associated with a set of possible feedback messages and the content of these messages are coded (= in advance structured feedback messages), the codes are elaborated and discussed. c) Technical instruction in using the BIE device. d) Modeling by the coach, while wearing the BIE device, of good teacher behavior. e) Practice time using the BIE device. 3) Discussing performance of the pre-service teacher: 4) Intervention: The actual usage of the BIE device in the classroom, preferably four times (Farrell & Chandler, 2008; Fry & Hin, 2006; Kahan, 2002; O’Reilly et al., 1994; Price, Martella, & Marchand-Martella, 2002; Rock et al., 2009b), with each lesson ending in a reflection. In this reflection, the next targeted behavioral improvement and next codes should be discussed. 5) Phasing out the usage of BIE (keeping the BIE in the ear but not receiving immediate performance feedback from the coach and by the end not using the BIE device at all). 6) Evaluation of performance of pre-service teacher. BIE device: The implementation of the synchronous coaching mode was realized through the use of a BIE device. This device is a Sennheiser DW8000 wireless transmitter with earpiece. The transmitter and receiver device was attached to the waistband, and the listening device was inserted into the ear (with connecting wires hidden under the pre-service teacher’s shirt). Immediate feedback was defined as feedback delivered by the coach within three seconds 61
Chapter 4 after the target behavior occurred (Scheeler, Macluckie, & Albright, 2008; Scheeler & Lee, 2002). This criterion was established in a separate study that looked at the latency between a pupil response and a teacher consequence (Scheeler & Lee, 2002). The questionnaire: The perceived satisfaction of the control and experimental groups were measured by use of a questionnaire with 18 Likert-scale items and two open questions (e.g., “What do you find positive/negative about the coaching you received?”) The operationalization of perceived satisfaction with the coaching mode is based on the five functions of coaching (Joyce & Showers, 1980, 1988). These functions are: 1) 2) 3) 4) 5)
giving supportiveness and confidence, giving feedback, discussing suggestions, discussing effects on pupils, and giving encouragement.
In a study by Showers (1984) these five functions served as criteria to assess the quality of (peer) coaching (Engelen, 2002). For each of these five functions, items were already formulated in a study by Engelen (2002). Four items addressed the first function, giving support and confidence (e.g., “Your coach showed respect for your opinion regarding good teaching”); six items inquired about the second function, giving feedback (e.g., “The given feedback was useful”); four items addressed the third function, discussing suggestions (e.g., “The suggestions were realistic”); three items were for the fourth function, discussing effects on pupils (e.g., You discussed the reactions on the changed approach of the pupils); and one item asked pre-service teachers to evaluate encouragement (Engelen, 2002). Respondents were required to rate each item on a Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). The questionnaire was developed in close cooperation with three coaches working in secondary education (pilot-study group), who looked at the item formulation and the appropriateness of the items (i.e., their face validity). All items had already been thoroughly tested with respect to their psychometric qualities (i.e. validity and reliability) in a study of Engelen (2002), who found respectively for the different functions of coaching following data: supportiveness (Cronbach alpha coefficient of .83), giving feedback (Cronbach alpha coefficient of .75), discussing suggestions (Cronbach alpha coefficient of .74), discussing effects on pupils (Cronbach alpha coefficient of .69), and encouragement (one item). Our findings were in line with these findings: supportiveness (Cronbach alpha coefficient of .67), giving feedback (Cronbach alpha coefficient of .85), discussing suggestions (Cronbach alpha coefficient of .72), discussing effects on pupils (Cronbach alpha coefficient of .78), and encouragement (one item). Pre-service teachers and coaches were instructed not to place their names on the questionnaires, and a box was placed in a common area for teachers to return the forms. After two reminders, every pre-service teacher had filled out the questionnaire.
62
Perceived satisfaction of pre-service teachers 4.4.3
Procedure
This study was carried out in a real-life setting of a classroom with pupils. Approval for the study was obtained from school principals, cooperating teachers (now coaches) and preservice teachers. First, pre-service teachers and coaches were asked to volunteer in an experiment on performance growth of pre-service teachers which would require a time investment of 10 hours. In exchange, it was promised that individual information on the performance of the pre-service teacher and the general results of the study would be provided to them. Next, they were informed of the premise of the study. Pre-service teachers would receive feedback on three teaching behavior areas: classroom management, activating instruction, and altering approaches towards different pupils. These three teaching behaviors are important for pre-service teachers and in line with the standards of Dutch teacher training. In a metaanalysis of 83 studies of the reported problems of pre-service teachers, Veenman (1984) found that pre-service teachers have similar problems with achieving effective classroom management, activating pupils towards their learning process, and catering to the individual needs of pupils. Recent literature supported these findings (Stroot et al., 1998; Houtveen, Versloot, & Groenen, 2006). Each pair of pre-service teacher and coach was equally and randomly assigned to a coaching method. The protocol for the organization of either asynchronous or experimental group was similar to and in line with the six-step approach for synchronous coaching (Coninx et al., 2013). After that, participants were assigned to either the control or experimental group, and the researcher scheduled a conference for each pair. The procedure explanation differed for the control group (six-step approach without immediate feedback) and for the experimental group (six-step approach): 1) Control group (six-step approach without immediate feedback). A coach provided feedback on teaching practice at three identified stages: a) pre-observation conference (steps 1, 2, and 3); b) observation (step 4); and c) post-observation conference (steps 5 and 6) (Rock et al., 2009a; Scheeler & Lee, 2002). This method provided feedback from experienced teachers to pre-service teachers in a delayed manner, where the experienced teachers acted as observers and attempted to remain unobtrusive and silent while taking notes in an effort to avoid interruption of the instructional flow in the classroom (Scheeler et al., 2006; West & Jones, 2007). In the Netherlands, this is a traditional way of coaching teachers. 2) Experimental group (six-step approach). A coach provided feedback on teaching practice at the same three identified stages. The coach was also sitting in the back of a classroom while observing the behavior of the pre-service teacher, but with one difference. While the pre-service teacher was teaching, a coach “whispered” on-the-spot feedback to the preservice teacher in an unobtrusive manner. Immediate feedback was defined as feedback delivered by the coach within three seconds after the target behavior had occurred (Scheeler et al., 2008; Scheeler & Lee, 2002). This criterion was established in a separate study that looked at the latency between a student response and a teacher consequence (Scheeler & Lee, 2002).
63
Chapter 4 The protocol for the organization of either control or experimental group was similar to and in line with the six-step approach for synchronous coaching (Coninx et al., 2013). After that, participants were assigned to either the control or experimental group, and the researcher scheduled a conference for each pair. Every pair of pre-service teacher and coach followed a similar meeting independently from the coaching modus. The experimental and control groups were given extra time for practicing with the equipment, identifying and responding to technical glitches, and discussing the code system. Codes were established: short phrases (i.e., keywords) that could indicate feedback on the three teaching behaviors (classroom management, activating instruction and differentiation). Table 4-1: Protocol for the control and experimental group according to the six-step approach of Coninx et al. (2013) Protocol for this study
Step 1 : Taking care of two prerequisites (willingness and approval) Step 2: Conference prior to the field experience 1. Explanation of the procedures and roles of coach and pre-service teacher 2. Explanation of the targeted behavior on classroom management, activating instruction and differentiation towards different students 3. Technical instruction in using a BIE device 4. Modeling good teacher behavior by coach 5. Practice time using a BIE device Step 3 : Discussing of measurements of performance pre-service teachers Step 4: Intervention (four times, preferably at the same time for four weeks) with pre-observation conference, observation by a coach of an actual lesson sitting in the back of a classroom, and post-observation conference in which targeted performance is discussed Step 5 : Phasing out the use of the BIE Step 6: Evaluation of performance of pre-service teacher Number of respondents
Control group (asynchronous coaching) N=15 idem
Experimental group (synchronous coaching) N=15
idem Idem
idem idem + compose ‘codes’ (= in advance structured feedback idem idem + modeling usage of BIE by coach idem idem idem + synchronous coaching
not applicable idem not applicable idem idem
not applicable idem 30 pre-service teachers and their coaches
idem
idem idem 30 pre-service teachers and their coaches
Pre-service teachers would create lessons plans, have them reviewed by the coach, and receive feedback from the coach prior to the teaching lessons. During teaching lessons, the coach would sit at the back of a classroom for observation. The pre-service teachers in the experimental group would receive feedback via BIE in addition to a post-observation conference. The control group would receive feedback via the traditional post-observation conference. After each weekly session, the coach and pre-service teacher were asked to make written notes about the effectiveness of their feedback method. Four weeks of observation and post-observation conferences were planned. Table 4-1 summarizes the steps. 4.4.4
Results
The data gathered through the administration of the questionnaires supported the findings of the literature search (as reported in the introduction). On average, the pre-service teachers from both control and experimental group are satisfied with their coaching modus. The preservice teachers are the most satisfied with the functions “giving feedback” and “encouraging.” For the functions “discussing suggestions” and “discussing effects on pupils,”
64
Perceived satisfaction of pre-service teachers they are on average satisfied. For the function “supportiveness,” pre-service teachers are less satisfied. Table 4-2: Mean and standard deviation of perceived satisfaction of pre-service teachers regarding the different functions of coaching Functions of coaching
Synchronous coaching (N=15) Asynchronous coaching (N=15) M SD M SD 1 supportiveness* 3.5 .49 3.0 .48 2 giving feedback* 4.0 .58 3.4 .71 3 discussing suggestions 3.5 .46 3.3 .40 4 discussing effects on pupils 3.7 .70 3.4 .71 5 encouragement* 4.3 .71 3.7 .82 (*) significant difference between control and experimental group for alpha 0.05
Results indicated that the synchronous coaching group showed a significantly greater satisfaction for three functions than did the asynchronous coaching group. Participants from the experimental group found their coaching more supportive, were more satisfied with the given feedback, and found their coaching to be more encouraging. The other two functions were also positive but not statistically more positive. The evaluation questionnaire indicated that although both groups of pre-service teachers had previously had equal time exposure to coaching, the experimental group was found to be significantly more positive on three functions, namely, supportiveness, giving feedback, and encouragement. For the first function “supportiveness,” an independent-samples t-test indicated a significant difference (t(N)=30, p=0.004) between the control group (M=3.0, SD=0.48), and the experimental group (M=3.5, SD=0.49). The magnitude of the difference in the means was large (Cohen’s d=1.07). Pre-service teachers experienced significantly more support in the experimental group (synchronous coaching). We find similar results in the written reflections of the pre-service teachers of the synchronous coaching group. “My coach gave me short instructions that were very supportive. I wasn’t standing alone in front of the class. It felt like a helping person standing next to me.” (Bart, pre-service teacher, 20 years old) For the second function “giving feedback,” an independent-samples t-test indicated a significant difference (t(N)=30, p=.015) between the control group (M=3.4, SD=.71), and the experimental group (M=4.0, SD=.58). The magnitude of the difference in the means was large (Cohen’s d=0.96). Pre-service teachers stated that they received significant better feedback in the experimental group (synchronous coaching). A closer look at the items for this scale shows that there was a significant difference between control and experimental group for useful feedback in general, specific feedback, and useful feedback for planning following lesson. The synchronous coaching group favorably stated that the giving of feedback was more useful, more specific and more useful for planning the following lesson than was the received feedback in the control group. We find similar results in the written reflections of the pre-service teachers of the synchronous coaching group: “I think it [synchronous coaching] is effective; you can do a lot more exercising during your field experience. You learn to master [teaching] more quickly because there are quite a lot of things you should pay attention to. With immediate feedback, I can practice more in the class.” (Leen, pre-service teacher, 21 years old)
65
Chapter 4 “I had a coach who observed my teaching performance, but it was very sporadic. For example, I have problems with classroom management with a certain class, and my coach has visited me only once. With synchronous coaching, a coach needs to be present in a class.”(Rik, pre-service teacher, 19 years old) “Because you received instructions in the moment, the post-conference debriefing is much shorter (I think). There is more awareness; if my coach would ask, ‘Do you remember this moment?’ I will probably have forgotten it. But if I hear during my lesson, ‘This is it,’ then I will remember.” (Fien, pre-service teacher, 20 years old) For the third function “encouragement,” an independent-samples t-test indicated a significant difference (t(N)=30, p=.029) between the control group (M=3.7, SD=.82) and the experimental group (M=4.3, SD=.71). The magnitude of the difference in the means was large (Cohen’s d=0.81). Pre-service teachers experienced significantly more encouragement in the experimental group (synchronous coaching). We find similar results in the written reflections of the pre-service teachers from the synchronous coaching group: “It was nice to be encouraged by my coach.” (Rik, pre-service teacher, 19 years old)
4.5 Conclusion and discussion Principles drawn from feedback theory indicate that, in order to learn specific tasks most efficiently, the learners must be informed frequently, immediately, and clearly about the adequacy of their performance. Synchronous coaching permits a coach to provide this kind of feedback at the moment when it is needed (Baum, 1976). The challenge of providing synchronous coaching in an efficient and effective manner provided a context of discussing teaching techniques (Domoto et al., 1979). The practical value is that it may help schools, school leaders, and teacher-education programs to view synchronous coaching as one element of a more comprehensive pre-service teacher-education program. One suggestion to improve pre-service teacher-education is to provide more direct in-class feedback than is frequently offered (Bowles and Nelson, 1976). By providing synchronous coaching to pre-service teachers, they receive on the spot behavioral alternatives for handling specific child misbehaviors (Crimmins, Bradlyn, St-Lawrence, & Kelly, 1984), which is the accepted goal of pre-service teachers’ education. The results of the in vivo study indicated that synchronous coaching is a satisfying technique for coaching pre-service teachers. Overall, pre-service teachers in the current study rated synchronous coaching favorably with respect to three functions of coaching, namely, supportiveness, giving useful feedback, and encouragement. This is consistent with the findings of the extensive literature search. However, there are some limitations of this study. The first has to do with the questionnaire. This questionnaire was based on the theoretical framework of D. Kirkpatrick and P. Kirkpatrick (2006), and the operationalization of satisfaction of the coaching was based on Joyce and Showers (1988). Although face-validity (based on teacher trainers in the field of secondary education) showed that the instrument is suitable for secondary education, future empirical research using different satisfaction scales is necessary to confirm our findings.
66
Perceived satisfaction of pre-service teachers
The second limitation of the study is that the design of the present study does not make it possible to ascertain if the addition of training programs of synchronous coaching enhanced perceived satisfaction. Also, perceived satisfaction may have been the result of the trainings program in tandem with synchronous coaching sessions, or of the synchronous coaching sessions alone (Bowles and Nelson, 1976). The third limitation had to do with the in vivo setting, with its lack of control, e.g., over what content areas were taught. While these limitations may have influenced the findings, the current study nevertheless provides an important indication for future research in the area of synchronous coaching and immediate performance feedback during the field experiences of pre-service teachers. Further research in the area synchronous coaching should include four domains. The first domain is that of investigating the maintenance of performance of pre-service teachers who have been coached synchronously. Most of the studies of synchronous coaching address the acquisition of specific behavior. It is also desirable to investigate whether this behavior still exists after a certain period (retention of the changed behavior). The second domain includes research on the influencing aspects of synchronous coaching. Although the theoretical categories were based on international literature, it would be interesting to identify other influencing factors of coaching. Replications involving different teaching subjects, different classes, and pre-service teachers of varying abilities and different school levels may be of interest. Thirdly, it is important to include coaches’ voices in future research to confirm or reject the positive satisfaction with synchronous coaching. Perhaps coaches are more satisfied with the asynchronous coaching. The fourth domain includes measuring the effects of synchronous coaching. Research on the effectiveness of a coaching program can be performed on four levels, according to D. Kirkpatrick and P. Kirkpatrick(2006): 1) level of reaction; 2) level of learning knowledge, skills, and attitudes; 3) level of behavior (change in behavior of preservice teacher); and 4) level of results (effects on pupils). For this study, we circumscribe the first level of reaction (Kirkpatrick, 2002) namely satisfaction. Further research should confirm if synchronous coaching will lead to more learning activities and eventually more effective teaching behavior. This study confirmed that pre-service teachers are satisfied with synchronous coaching, but have these teachers changed their teaching behavior? This question will be addressed in future research. In conclusion, results of this study suggest that the unique niche filled by synchronous coaching during pre-service teachers’ field experiences coaches to provide immediate performance feedback, which is a valuable tool, based on the satisfaction of the pre-service teachers.
References Baum, D. (1976). An application of the “bug-in-the ear” communication system for training psychometrics. Counselor Education and Supervision, 15, 309–310. Billingsley, B.S., Carlson, E., & Klein, S. (2004). The working conditions and induction support of early special career special educators. Exceptional Children, 70(3), 333– 347. Boe, E.E., Cook, L.H., & Sunderland, R.J. (2008). Teacher turnover: Examining exit attrition, teaching area transfer, and school migration. Exceptional Children, 75(1), 7–31.
67
Chapter 4 Bowles, E.P., & Nelson, R.O. (1976). Training teachers as mediators: Efficacy of a workshop versus the bug-in-ear technique. Journal of School Psychology, 14(1), 15–25. Brown, S.W. (2005). Emily and Rebecca: A tale of two teachers. Teaching and Teacher Education, 21(6), 637–648. Brownell, M.T., Hirsch, E., & Seo, S. (2004). Meeting the demand for highly qualified special education teachers during severe shortages: What should policymakers consider? Journal of Special Education, 38(1), 56–61. Chow, B.C., Mak, D.K.C., Cheung, S.Y. & Louie, L.H.T. (2001). Sharing of good practice: Use of the transmitter-earning. Hong Kong: Hong Kong Baptist University. Clutterbuck, D. (2004). Everyone needs a mentor: Fostering talent at work. London: CIPD. Crimmins, D.B., Bradlyn, A.S., St-Lawrence, J.S., & Kelly, J.A. (1984). A training technique for improving the parent-child interaction skills of an abusive neglectful mother. Child Abuse and Neglect, 8, 533–539. Coninx, N. S., Kreijns, C. J., & Jochems, W. M. G. (2013). The use of keywords for delivering immediate performance feedback on teacher competence development. European Journal of Teacher Education, 36(2), 164–182. Coninx, N. S., Kreijns, C. J., & Jochems, W. M. G. (2013). A review of synchronous coaching using BIE device. Submitted. Darling-Hammond, L. (2000). How teacher education matters. Journal of Teacher Education, 51(3), 166–73. Darling-Hammond, L., & Bransford, J. (with LePage, P., Hammerness, K., & Duffy, H.). (2005). Preparing teachers for a changing world: What teachers should learn and be able to do. San Francisco: Jossey-Bass. Domoto, P.K., Weinstein, P., & Getz, T. (1979). A pilot study using remote broadcasting equipment to provide instruction in pedodontics. Journal of Dentistry Education, 43, 599–601. Engelen, A.J.A. (2002) Coaching binnenstebuiten: een onderzoek naar coaching van docenten door docenten [Coaching from the inside out: a survey of coaching of teachers by teachers]. Unpublished doctoral dissertation: Utrecht, University Press. Farrell, A.C., & Chandler, D. (2008). Cooperating teachers’ impressions of the whisper in my ear (WIME) and traditional communication feedback methods for physical education pre-service teachers. Education and Human Development, 2(1), 1–9. Feiman-Nemser, S. (2001). From preparation to practice: Designing a continuum to strengthen and sustain teaching. Teacher College Record, 103, 1013–1055. Fry, J.M., Hin, M.K.T. (2006). Peer coaching with interactive wireless technology between student teachers: satisfaction with role and communication. Interactive learning environments, 14(3), 193–204. Giebelhaus, C. R., & Cruz, J. (1992). The third ear mechanical device: A supervision alternative. Journal of Early Childhood Teacher Education, 42(13), 8–12. Giebelhaus, C. R., & Cruz, J. (1994). The mechanical third ear device: An alternative to traditional student teaching supervision strategies. Journal of Teacher Education, 45, 365–373. Giebelhaus, C. R. & Bowman, C. L. (2002). Teaching mentors: Is it worth the effort? Journal of Educational Research, 95(4), 246–254. Goodman, J.I., & Duffy, M.L. (2007). Using bugs to increase student participation. Teaching exceptional children plus, 3(4), 365-37. Retrieved from: http: //scholarship.bc.edu /education/tecplus/vol3/iss4/art3. Goodman, J.I., Brady, M.P., Duffy, M.L., Scott, J., & Pollard, N.E. (2008). The effects of ‘bug-in-ear’ supervision on special education teachers’ delivery of learn units. Focus on autism and other developmental disabilities, 23 (4), 207–216.
68
Perceived satisfaction of pre-service teachers Hashweh, M.Z. (2003). Teacher accommodative change. Teaching and Teacher Education, 19, 421–434. Houtveen, T., Versloot, B., & Groenen, I. (2006). De begeleiding van beginnende leraren. [Coaching of beginning teachers]. Utrecht, Sectiebestuur Onderwijsarbeidsmarkt (SBO). Hunt, D.D. (1980). Bug-in-the-ear technique for teaching interview skills. Journal of Medical Education, 55, 964–966. Joyce B. and Showers B. (1980). Improving inservice training: the messages of research. Educational Leadership, 37(5), 379–385. Joyce, B. & Showers, B. (1988). Student achievement through staff development. New York: Longman. Kahan, D. (2002). The effects of a bug-in-the-ear device on intra lesson communication between a student teacher and a cooperating teacher. Journal of Teaching in Physical Education, 22(1), 86–104. Kersaint, G., Lewis, J., Potter, R., & Meisels, G. (2007). Why teachers leave: Factors that influence retention and resignation. Teaching and Teacher Education, 23(6), 775–794. Kirkpatrick, D. & Kirkpatrick, P. (2006). Evaluating training programs (3rd ed.). San Francisco, CA: Berrett-Koehler Publishers. Neukrug, E. (1991). Computer-assisted live supervision in counselor skills training. Counselor Education and Supervision, 31, 132–138. Noell, G. H., Duhon, G. J., Gatti, S. L., & Connell, J. E. (2002). Consultation, follow-up, and behavior management intervention implementation in general education. School Psychology Review, 3, 217–234. O’Reilly, M. F., Renzaglia, A., & Lee, S. (1994). An analysis of acquisition, generalization and maintenance of systematic instruction competencies by pre-service teachers using behavioral supervision techniques. Education and Training in Mental Retardation and Developmental Disabilities, 29, 22–33. Price, A. T., Martella, R. C., & Marchand-Martella, N. (2002). A comparison of immediate feedback delivered via an FM headset versus delayed feedback on the inappropriate verbalizations of a student with ADHD. Education and Treatment of Children, 25, 159–171. Rathel, J.M., Drasgow, E. & Christle, C.C. (2008). Effects of supervisor performance feedback on increasing pre-service teachers’ positive communication behavior with students with emotional and behavioral disorders. Journal of Emotional and Behavioral Disorders. 16(2), 67–77. Rock, M.L., Gregg, M., Thead, B.K., Acker, S.E., Gable, R.A., Zigmond, N.P. (2009). Can you hear me now? Evaluation of an online wireless technology to provide real-time feedback to special education teachers-in-training. Teacher Education and Special Education, 32(1), 64–82. Rock, M.L., Gregg, M., Howard, P.W., Ploessl, D.M., Maughn, S., Gable, R.A., Zigmond, N.P. (2009). See me, hear me, coach me: Virtual bug-in-ear technology brings immediacy to professional development. National Staff Development Council, 30(3), 24–32. Rushton, S. P. (2003). Two pre-service teachers’ growth in self-efficacy while teaching in an inner-city school. The Urban Review, 35(3), 167–189. Scheeler, M.C., Lee, D.L. (2002). Using technology to deliver immediate corrective feedback to preservice teachers. Journal of behavioral education, 11(4). 231–241. Scheeler, M.C., Ruhl, K.L., McAfee, J.K. (2004). Providing performance feedback to teachers: A review. Teacher Education and Special Education, 27(3), 396–407.
69
Chapter 4 Scheeler, M.C., McAfee, J.K., Ruhl, K.L., Lee, D.L. (2006). Effects of corrective feedback delivered via wireless technology on preservice teacher performance and student behavior. Teacher Education and Special Education, 29(1), 12–25. Scheeler, M.C., Macluckie, M., Albright, K. (2008). Effects of immediate feedback delivered by peer tutors on the oral presentation skills of adolescents with learning disabilities. Remedial and Special Education, 11, 1–10. Showers B. (1984). Peer coaching: A strategy for facilitating transfer of training. Eugene, OR: Center for Educational Policy and Management. Smith, T. M., & Ingersoll, R. M. (2004). What are the effects of induction and mentoring on beginning teacher turnover? American Educational Research Journal, 41(3), 681–714. Smith, D. L. & Thompson, S. (2005). Creating highly qualified teachers for urban schools. Professional Educator, 27, 73–88. Stroot, S., Keil, V., Stedman, P., Lohr, L., Faust, R., Schincariol-Randall, L., Richter M. (1998). Peer assistance and review guidebook. Columbus, OH: Ohio Department of Education. Swafford, J. (1998). Teachers supporting teachers through peer coaching. Support for Learning, 13(2), 54–58. Veenman, S. (1984). Perceived problems of beginning teachers. Review of Educational Research, 54(2), 143–178. Wade, Y.W. (2010). Increasing novice teacher support in 21st century classrooms: Induction and mentoring for beginning teachers through bug-in-ear technology. Unpublished doctoral thesis. Orland. Florida : University of Central Florida. Ward, C. (1960). An electronic aid for teaching interview techniques. Archives of General Psychiatry, 3(4), 357–358. Ward, C. (1962). Electronic preceptoring in teaching beginning psychotherapy. Journal of Medical Education, 37(12), 1128–1129. West, E., & Jones, P., (2007). A framework for planning technology used in teacher education programs that serve rural communities. Rural Special Education Quarterly, 26(4), 3– 15. White, J., & Gordon, J. (2000). There’s a bug in my ear: Value adding through authentic interview experience. Contemporary Issues in Early Childhood, 2(1), 287–298. Wong, H.K., Britton, T., Gasner, T. (2004). What the world can teach us about improving new teacher induction. Phi Delta Kappan, 86(5), 379–384. Woullard, R. & Coats, L. T. (2004). The community college role in preparing future teachers: The impact of a mentoring program for pre-service teachers. Community College Journal of Research and Practice, 298, 609–624.
70
Chapter 5 Observed behavioral change of pre-service teachers5 Abstract: Review of current literature on feedback revealed that when it is specific, immediate, and goal-oriented, the feedback will be effective on (pre-service) teacher’s performance. Synchronous coaching using a BIE device can provide this kind of feedback.We developed a systematic approach for using synchronous coaching. The aim of the present study was to explore whether synchronous coaching affects the behavior of pre-service teachers. To this end, a literature search was conducted on behavioral change of pre-service teachers who received synchronous coaching. An in vivo study with 1197 pupils and a total of 80 pre-service teachers and their coaches (40 pairs) supported the literature findings that synchronous coaching is more effective in positively changing the behavior of pre-service teachers than is asynchronous coaching. In particular we found that pre-service teachers became more cooperative and had more ascendancy in their classroom than pre-service teachers who received asynchronous coaching. These findings and other considerations for additional research using synchronous coaching are addressed. Keywords: synchronous coaching, immediate specific performance feedback, BIE device, WIME device, behavioral change, pre-service teachers, QTI
5.1 Introduction Pre-service teachers seem not well prepared for their teaching job. A number of studies show that pre-service teachers may experience feelings of anxiety, isolation and lack of support (Brown, 2005; Rushton, 2003; Smith & Thompson, 2005; Woullard & Coats, 2004). An increased number of pre-service teachers describe feelings of unpreparedness for general education classrooms and even less prepared for divers students (Brownell, Hirsch & Seo, 2004). In a meta-analysis of 83 empirical studies of the reported problems of pre-service teachers, Veenman (1984) found that pre-service teachers have problems with achieving effective classroom management, activating pupils towards their learning process, and catering for individual needs of pupils. Recent literature supports these findings (Stroot, Keil, Stedman, Lohr, Faust, Schincariol-Randall, Sullivan, Czerniak, KKuchcinski, Orel & Richter, 1998; Houtveen, Versloot & Groenen, 2006; Helms-Lorenz, Slof, Vermue & Canrinus, 2012; Houtveen & Van de Grift, 2012). Teacher preparation programs are challenged to improve pre-service teachers’ preparation (Billingsley, Carlson & Klein, 2004; Boe, Cook & Sunderland, 2008; Kersaint, Lewis, Potter & Meisels, 2007). Hasweh (2003) called for more systematic and empirically validated research that is directed towards the support and optimization of teacher education programs. This study aimed to contribute to teacher education programs to view synchronous coaching 5
This chapter is submitted for publication as an article titled “Observed behavioral change of pre-service teachers”.
71
Chapter 5 as one element of a more comprehensive pre-service teacher education program to support pre-service teachers with competences they will use in their professional live. Being not well prepared can lead to quitting their professional career in education. High rates of turnover contribute significantly to the attrition of beginning teachers in education (Darling-Hammond, 2000; Smith & Ingersoll, 2004). Facing teacher shortages, teacher preparation programs have gained increased attention as a means of assisting pre-service teachers’ successful entry into the teaching profession (Giebelhaus & Bowman, 2002). Researchers have focused on efficient and effective means for preparing pre-service teachers who can reach the higher standards imposed by the changing society and who will be kept for the teaching profession (Darling-Hammond & Brandsford, 2005; Helms-Lorenz, Slof, Vermue & Canrinus, 2012). Coaching has proven to be an effective way of providing feedback to pre-service teachers in order to improve teachers’ instructional effectiveness (Feiman-Nemser, 2001). Scheeler, Ruhl and McAfee (2004) provided evidence that teacher instruction improved when a coach gave specific, immediate and goal oriented performance feedback to pre-service teachers. Teachers — as other learners — acquire and maintain new behaviors best when they receive systematic feedback, have multiple opportunities to practice and receive performance feedback that is specific, immediate and goal oriented through synchronous coaching. Therefore, our hypothesis is: Synchronous coaching is more effective than traditional, asynchronous coaching with respect to changing teaching behavior of pre-service teachers. In order to test this hypothesis we conducted an explorative study in which we compared the performances of an experimental group receiving synchronous coaching and a control group receiving asynchronous coaching. A literature search was performed, which lead to an understanding of synchronous coaching and gave a basis for the theoretical framework. Then the method used is described. In the last section the conclusions are drawn and discussed for future research possibilities.
5.2 Synchronous coaching Preparing pre-service teachers for diverse populations of pupils is an important task that must be addressed systematically. Current teacher education literature points for the need for standards and accountability and stresses the importance of well-researched, teacher preparation methods (Goodman & Duffy, 2007). Coaches can provide specific goal oriented feedback in two ways, namely delayed or immediate (Scheeler, McAfee, Ruhl & Lee, 2006; O’Reilly, Renzaglia & Lee, 1994). In delayed feedback, coaches take notes while observing a lesson and share these notes with the pre-service teacher upon lesson completion, perhaps even later. However, post-observation conferences (i.e., discussions between coach and pre-service teacher after completion of the lesson) are most effective when they occur immediately after the observation (Swafford, 1998). The disadvantage with delayed feedback, however, is that the pre-service teacher may practice less effective methods, offering pupils less effective instruction, thus causing them to possibly acquire ineffective or inaccurate skills. In addition, delayed feedback makes it difficult for both coach and pre-service teacher to reconstruct personal examples of what was occurring in the targeted teaching interchange thereby
72
Observed behavioral change of pre-service teachers reducing the feedback’s effectiveness (Scheeler et al., 2006). Immediate feedback does not have this disadvantage. In immediate feedback, coaches who observe a problem in pre-service teacher behavior or an inappropriate behavior instantly can stop the instructional activity and correct the pre-service teacher’s behavior directly during the observation (Scheeler et al., 2006). Immediate performance feedback allows coaches to provide pre-service teachers with immediate, on-thespot feedback while these pre-service teachers are engaging in classroom instruction (Giebelhaus & Cruz, 1992, 1994). Therefore, immediate feedback, although interrupting and disturbing class-room instruction, is mostly preferred. But, the interruption and disturbance of immediate feedback can be avoided by the use of technology as it enables unobtrusive immediate feedback while pre-service teachers receive real-time, on-the-spot feedback during classroom experiences. Although researchers have used various forms of technology to provide immediate feedback, the Bug In Ear (BIE) is the most commonly used device. Researchers have variously referred to this technology as a ‘whisper-in-my-ear (WIME)’, ‘wireless earphone’, a ‘mechanical third ear device’ or an ‘electronic audio-cueing system’ (Rock, Gregg, Thead, Acker, Gable & Zigmund, 2009a). Most of us know BIE device as the tiny earpieces that people wear while they talk on their cell phones. BIE device refers to a small ear bud receiver that transmits verbal communication from a sender to a receiver. Synchronous coaching is a method for giving immediate performance feedback with certain characteristics (specific, corrective, encouraging) by using a BIE device on targeted performance. In order to minimize disturbances, a synchronous coaching protocol is used in the actual classroom (described in more detail in the section 1.4.3). This protocol entails that before the lessons start, the target behavior of the performance feedback and the associated set of possible feedback messages is determined in consultation with the pre-service teacher. As the contents of these messages are coded through keywords (for the purpose of eliminating any ambiguity and reducing cognitive load), the meaning of these keywords needs to be elaborated and discussed in advance. In the most common setting, a coach or a colleague sits in the back of an actual class room and when appropriate sends keywords to the pre-service teacher. Technically, sending a keyword is accomplished by pressing the right button on a computer interface that as a reaction will use a computerized voice to transmit the keyword (Coninx, Kreijns & Jochems, 2013). The literature review suggested that although there has been research involving the support of synchronous coaching for pre-service teachers there is only limited research regarding the link between a systematic approach of synchronous coaching and its effectiveness. Besides, no research is available with respect to keyword-based synchronous coaching as proposed here. Consequently, there is a need to evaluate the potential effect of keywords-based synchronous coaching to pre-service teachers.
5.3 Theoretical framework The best known evaluation methodology of training programs is Donald Kirkpatrick’s ‘Four Level Evaluation Model’ (Kirkpatrick & Kirkpatrick, 2006). His model describes four measures in evaluating a training program: 1) Reaction: a measure how participants feel about the various aspects of a training program, i.e. a measure for customer satisfaction; management often makes decisions about a 73
Chapter 5 training program based on participants’ comments’ to ensure that participants are motivated and interested in learning. If they don’t like the program, there is little chance that they will put any effort in it. 2) Learning: a measure of the knowledge acquired, skills improved or attitudes changed due to training; generally, a training program accomplishes one or more of those three things. 3) Behavior: a measure of the extent to which participants change their on-the-job behavior because of training. 4) Results: a measure of the final results that occur due to training, including increased sales, higher productivity, reduced costs, less employee turnover and improved quality. For the purpose of the present study we have focused on Kirkpatrick’s third level of evaluation of a training program (i.e., the focus was on teachers’ behavioral change), as we were interested in developing on-the-job-behavior of pre-service teachers. In order to measure the effectiveness of synchronous coaching with the use of BIE device on Kirkpatrick’s third level, the areas of focus for the development of pre-service teacher professional behavior is needed to be described. Next, a way of measuring is defined. A systematic literature search on synchronous coaching regarding behavior changes during the training of pre-service teachers was conducted. 5.3.1
Literature search
The literature search that considered the level of behavioral change (Kirkpatrick & Kirkpatrick, 2006) was conducted. These articles were analysed for giving an answer on the following question: ‘To what extent do empirical articles report effects on Kirkpatrick’s level of behavioral change?’ The ERIC search used (a combinations of) the following terms: ‘immediate feedback’, ‘performance feedback’, ‘bug-in-ear’, ‘wireless communication’ and ‘teachers’. Articles were selected if they included one or more of these terms. Articles including the above search terms combinations in their title or abstract and focusing on immediate feedback were included for analysis. The ‘snow ball’ method was applied by examining the references of the articles which led to other relevant articles. The raw yield was 61 articles. These were further examined on additional criteria including the clarity of the described method section and the use of an actual BIE device. Studies in which the researchers used immediate e-mail after the lessons (Rathel, Drasgow & Christle, 2008) or researchers who used keyword entry placed on a monitor (Neukrug, 1991) were excluded due to the lack of a BIE device usage. Additional excluded papers were either offtopic or poorly designed. On the basis of these criteria the netto yield was 29 articles6. Of these studies, 19 studies focused on behavior change.
6
The literature search revealed that most of the studies (23 of the 29 studies) were carried out in the education sector with 49% in primary education, 26% in special education and 17% in secondary school education. Also, the majority of the studies (18 of the 29 studies) reported qualitative data. Twelve of the 29 studies made use of statistical analyses and, therefore, can be qualified as quantitative. Of the 29 studies only 8 studies used an experimental comparison between a control group and an experimental group. The numbers of respondents were relatively low, going from a single subject to 34 respondents. White and Gordon (2000) were an exception with a study of 60 participants.
74
Observed behavioral change of pre-service teachers 5.3.2
Situational context of teacher behavior
The study of Coninx, Kreijns and Jochems (2013) showed that the content of the behavioral changes encompasses three categories: classroom management, instructional activation of pupils and pedagogical relationship between teacher and pupils. For each of these categories, we give an example that illustrates its contents. Classroom management. The study of Bowles and Nelson (1976) with 13 pre-service teachers demonstrated that the experimental group (synchronous coaching) showed a better classroom management than the control group (no synchronous coaching). BIE-technology was used. The teacher wore a portable FM radio receiver. The experimenter used a wireless FM microphone to broadcast instructions to the teacher while performing usual activities in the classroom. For classroom management, a significant difference was observed with respect to two behaviors: encouragement and prompting. For other specific behaviors, no significant difference was found. Instructional activation of pupils. In the study of Giebelhaus (1994) with 22 pre-service teachers a significant difference was found on instructional activation of pupils for the experimental group (synchronous coaching) in comparison with the control group (no synchronous coaching). Instructional activation of pupils is defined in 14 clarity behaviors. Clarity behaviors are discrete teacher clarity skills documented in the literature as observable and measurable. For example ‘repeat things that are important’ with the audio cue: ‘repeat;’ or ‘work through examples on the board and demonstrates the procedures to be used’ with the audio cue: ‘demonstrate’. Only one of the 14 clarity behaviors showed a significant difference namely ‘ask questions’. The pre-service teachers of the experimental group did ask significant more questions to pupils than the pre-service teachers of the control group. Pedagogical relationship with pupils. The study of Rock and colleagues (2008) with 15 preservice teachers examined whether there is a significant difference on class room climate due to two conditions (with synchronous coaching vs. without synchronous coaching). Classroom climate is defined as the use of encouragement, redirects and reprimands and the number of pupils who are committed in a lesson. There was a significant difference between the two conditions. More pupils were committed in a lesson where a pre-service teacher was synchronously coached (experimental group) as compared to where it was not the case (control group). 5.3.3
Operationalization of behavior
The operationalization of effective teacher behavior is derived from a system perspective. Researchers are convinced that one of the real keys to effective teaching behavior lies in the nature of the interpersonal behavior of the teacher (Wubbels & Levy, 1993). Wubbels, Creton and Holvast (1988) investigated teacher behavior in classrooms from a systems perspective, adapting a theory on communication processes developed by Watzlawick, Beavin and Jackson (1967). Within the systems perspective on communication, it is assumed that the behaviors of participants influence each other mutually. The behavior of the teacher is influenced by the behavior of the pupils and in turn influences student behavior. Circular communication processes develop which not only consist of behavior, but determine behavior as well.
75
Chapter 5
Greer (2002) calls this mutual influence system a learning unit (LU) in which he distinguishes three elements: a single behavior act of a pre-service teacher, a reaction of a student and then an adjusted single response of the pre-service teacher. A LU is an interaction between a teacher and a student and is the smallest unit in which a particular teacher behavior can be displayed. LU’s can be observed and be measured. If a LU is set and discussed, specific feedback messages can be constructed for use in a synchronous coaching session. With the systems perspective in mind, Wubbels, Creton, and Hooymayers (1992) developed a model to map interpersonal teacher behavior extrapolated from the work of Leary (1957). Leary concluded that a person’s interpersonal behavior could be described in two dimensions, which he named Proximity and Influence. The Proximity dimension can be depicted on a continuum that has cooperative behavior at one end and oppositional behavior at the other; the Influence dimension on a continuum with dominant behavior at one end and submissive behavior at the other. Leary plotted a person’s interpersonal behavior on a diagram that had the dominant/ submissive continuum (DS) as the vertical axis and the cooperative/oppositional continuum (CO) as the horizontal axis. In the application of the Leary model, teacher behavior is mapped on a Proximity dimension (Cooperation, C - Opposition, O) and an influence dimension (Dominance, D - Submission, S) resulting in four quadrants. In each quadrant has a predetermined dominance for one of the two factors to form eight sectors, each describing different behavior aspects: Leadership, Helpful/Friendly, Understanding, Student Responsibility and Freedom, Uncertain, Dissatisfied, Admonishing and Strict behavior. The description of each aspect can be found in Table 5-1. Table 5-1 Description of items for each scale of interpersonal behavior of pre-service teachers Scale Leadership [DC] Helping/ Friendly [CD] Understanding [CS] Pupil Responsibility/Freedom [SC] Uncertain [SO] Dissatisfied [OS] Admonishing [OD] Strict [DO]
Description Extent to which teacher provides leadership to the class and holds pupil attention. Extent to which the teacher is friendly and helpful towards pupils Extent to which the teacher shows understanding and care to pupils. Extent to which the pupils are given opportunities to assume responsibilities for their own activities Extent to which the teacher exhibits her/his uncertainty Extent to which the teacher shows unhappiness/dissatisfaction with the pupils. Extent to which the teacher shows anger/temper and is impatient in the class Extent to which the teacher is strict with demands of the pupils
In their application of the model to the classroom situation, Wubbels and collegues (1992) converted each octant of the original model to teacher behavior, each describing different aspects of interpersonal behavior. The sectors are labelled DC, CD and so on according to their position in the coordinate system, the letters coding the relative influence of the axes. For example, sectors DC and CD are both characterised by Dominance and Cooperation, but in DC Dominance predominates over Cooperation, whereas in CD Cooperation is more evident. The closer two sectors are to each other, the more similar are the teacher behaviors they represent. For a more detailed explanation of the model, the reader is referred to Wubbels, Brekelmans, and Hooymayers (1991).
76
Observed behavioral change of pre-service teachers 5.3.4
Questionnaire on Teacher Interaction
The Questionnaire on Teacher Interaction (QTI) is based on the model of Wubbels, Creton, and Hooymayers (1992). The original version of the QTI developed in the early 1980s in the Netherlands had 77 items (Wubbels, et.al., 1992). Later, an American version of the QTI was developed that had 64 items (Wubbels & Levy, 1991). The Australian version of the QTI used in this study is more economical with only 48 items. The latter QTI is based on the Wubbels model with the items allocated to eight scales, each corresponding to one of the eight sections in the model. Examples of items are "This teacher is friendly" (Helping/Friendly) and "This teacher gets angry unexpectedly" (Admonishing). The higher a score, the more a teacher shows behaviors from that sector. The use of the QTI in the Netherlands involved investigation of relationships between perceptions on the QTI scales and student outcomes (Wubbels, et.al., 1991). Regarding students’ cognitive outcomes, the more teachers demonstrated Strict, Leadership and Helping/ Friendly behaviors, the higher the cognitive outcomes scores were. Conversely, Student Responsibility/Freedom, Uncertain, and Dissatisfied behaviors were negatively related to achievement. Variations in the students’ appreciation of the subject and the lessons have been characterised on the basis of the Proximity dimension: the more cooperative the behavior displayed, the higher the affective outcome scores (Wubbels et.al., 1991). That is, Student Responsibility/Freedom, Understanding, Helping/Friendly, and Leadership behaviors were positively related to student attitudes. Uncertain, Dissatisfied, Admonishing, and Strict behaviors were negatively related to student attitudes. Overall, Brekelmans, Wubbels, and den Brok (2002) have indicated that teacher interpersonal behavior is an important aspect of the learning environment and that it is strongly related to student outcomes. So, in this study effective teaching behavior is defined in terms of getting higher on the Proximity dimension (C - O) and the Influence dimension (D - S). The hypothesis tested is that with immediate feedback (synchronous coaching) pre-service teachers will scored higher on the Proximity dimension and the Influence dimension than in the asynchronous way of coaching (control group).
5.4 Research method We conducted a study in a real-life setting with an experimental group (synchronous coaching) and control group (asynchronous coaching), based on the findings of the literature search. 5.4.1
Participants
Forty pre-service teachers were recruited from a teacher training institute for lower secondary education in southern Netherlands and ad random equally (twenty in each group) assigned to the control group or the experimental group. The pre-service teachers were asked to find a volunteering coach. All forty coaches had previously served as a coach of pre-service teachers and were experienced teachers. All coaches received positive recommendations from former pre-service teachers as being motivated and highly skilled professionals. Coaches received permission from necessary supervisors to coach pre-service teachers for the spring semester. A pre-service teacher together with a volunteering coach formed a pair.
77
Chapter 5 However, from the forty pairs ten pairs dropped-out (five in each group) due to sudden unavailability of the coach (2), sickness of the pre-service teacher (1), sickness of the coach (2), no suitable classroom (1) and not functioning BIE material (4). So, 60 participants volunteered with fifteen pairs in the control group and fifteen pairs in the experimental group. Thirty pre-service teachers (17 males; 13 females) aged 17-34 years (M=21.7, SD=3.8) taught different subjects, namely geography (10), history (5), Dutch (3), English (3), psychics (2), chemistry (3), and mathematics (4) to their pupils. 5.4.2
Instruments
Three instruments are used in this research: 1) The six step approach to synchronous coaching and 2) the use of BIE-technology, and 3) the QTI. The six step approach: On the basis of a previous extensive literature search of Coninx, Kreijns and Jochems (2013) a systematic approach of synchronous coaching was developed. This is a six steps approach with a training program for sending immediate performance feedback. These six steps are summarized as follows: 1) Taking care of two prerequisites a) Both coach and pre-service teacher must be willing to participate. b) Getting parental permission if pupils in the class room are being filmed or interviewed for research purposes. 2) Implementing a training program for using a BIE device consisted of five sub steps: a) Explanation of the procedures and roles of coach and pre-service teacher b) Determination of the target of the performance feedback, that is, which behavior should be trained? As each behavior is associated with a set of possible feedback messages ((i.e., the keywords) these keywords are elaborated and discussed. c) Instruction in the technical aspects of the BIE device. d) Modelling good teacher behavior while using BIE device by the coach e) Practicing BIE device usage. 3) Discussing performance of the pre-service teacher 4) Intervention: the actual usage of the BIE device in the class room: preferable four sessions should take place (Farrell & Chandler, 2008; Fry & Hin, 2006; Kahan, 2002; O’Reilly et al., 1994; Price, Martella & Marchand-Martella, 2002; Rock, Gregg, Howard, Ploessl, Maughn, Gable & Zigmund, 2009b) with each lesson ending in a reflection. In this reflection the next targeted behavioral improvement and next codes should be discussed. 5) Fading the usage of BIE (keeping BIE in the ear but not giving immediate performance feedback by coach and at the end not using the BIE device at all). 6) Evaluation of performance of pre-service teacher after four sessions of synchronous coaching. BIE-technology: The implementation of the synchronous coaching mode was realized by use of a BIE device. This device is a Sennheiser DW8000 wireless transmitter with earpiece. The transmitter and receiver device hung from the waistband and the listening device was inserted into the ear (with connecting wires hidden under the pre-service teacher’s shirt). Immediate feedback was defined as feedback delivered by the coach within three seconds after the target behavior occurred (Scheeler, Macluckie & Albright, 2008; Scheeler & Lee, 2002). This criterion was established in a separate study that looked at the latency between a pupil response and a teacher consequence (Scheeler & Lee, 2002).
78
Observed behavioral change of pre-service teachers QTI: To evaluate changes in interpersonal teaching behavior we used the QTI (Questionnaire on Teacher Interaction) developed by Wubbels et al. (1985). 5.4.3
Procedure
This study was carried out in a real life setting of a classroom with pupils. Approval for the study was obtained from school principals, cooperating teachers (now taking the role of coaches) and pre-service teachers. First, pre-service teachers and coaches were asked to volunteer in an experiment on coaching of pre-service which would need a time investment of ten hours. In exchange, it was promised that individual information on the performance of the pre-service teacher and the general results of the study will be provided to them. Next, they were informed of the premise of the study. Pre-service teachers would receive feedback on three teaching behaviors areas: classroom management, activating instruction and differentiation towards different pupils. Each pair of a pre-service teacher and a coach was randomly assigned to a coaching method, synchronous or asynchronous. The protocol for the organization of either experimental or control group was similar and in line with the six steps approach for synchronous coaching (Coninx et al., 2013). After assigning participants to either the control or experimental group, the researcher scheduled a conference for each pair. The procedure explained differed for either the control group (six-step approach without immediate feedback) or the experimental group (six-step approach) on one variable, namely the use of BIE. This way, the methods were equally designed. So, potential differences between the two methods can be explained by the use of BIE. The control group (six-step approach without immediate feedback) had the following treatment: a coach provided feedback on teaching practice at three identified stages: a) Pre-observation conference (steps 1, 2 and 3); b) Observation (step 4) c) And post-observation conference (step 5 en 6) (Rock et al., 2009a; Scheeler & Lee, 2002). This method provided feedback by experienced teachers to pre-service teachers in a delayed manner where the experienced teachers acted as an observer and attempted to remain unobtrusive and silent while taking notes in an effort to avoid interruption of instructional flow in the classroom (Scheeler et al., 2006; West & Jones, 2007). The experimental group (six-step approach) had a slight different treatment: a coach provided feedback on teaching practice at the three combined stages, used in the control group. The coach was also sitting in the back of the classroom while observing the behavior of a preservice teacher but with one difference. While a pre-service teacher was teaching, a coach ‘whispered’ on-the-spot feedback towards the pre-service teacher. This is accomplished by pressing buttons on the computer which launched pre-recorded feedback messages. For example for the feedback message: “to make eye contact with students”, the feedback message “eye contact” was pre-recorded en when pressing the button a pre-recorded (female) voice whispered ‘eye contact’ to the pre-service teacher. Every pair of pre-service teacher and coach followed a similar kick off meeting independently from the coaching modus. The experimental gets extra time for practicing with the equipment, identifying and responding to technical glitches and discussing the code system used. Keywords were used to establish short phrases that could indicate feedback on the three
79
Chapter 5 teacher teaching behaviors (classroom management, activating instruction and differentiation). In each case, the pre-service teachers ‘took over the lessons of the coaches’ while they were sitting in the back. The researchers have chosen for this approach because there is some kind of trust between coach and pre-service teacher (Wong, Britton & Gasner, 2004). The thirty coaches (8 males; 22 females) aged 28-54 years (M=41.4, SD=8.7) were working at eleven secondary schools, situated in southern and central parts of the Netherlands. The coaches had 3 to 20 years (M=8.5, SD=4.7) of experience in coaching pre-service teachers (Clutterbuck, 2004). Before the experiment started, each pre-service teacher administered the QTI in his or her class. Six hundred and thirty-nine (639) pupils completed the questionnaire. After four sessions (see procedure), the QTI was administered a second time; this time 558 pupils filled in the questionnaire. The pupils aged between 12 and 15 years. The number of pupils in a class ranged from 11 to 31 students. Pre-service teachers created lessons plans, have them reviewed by the coach and received feedback from the coach prior to the teaching lessons. During teaching lessons the coach was sitting in the back of a classroom for observation. The pre-service teachers of the experimental group received feedback via the BIE device in addition to a post-observation conference. The control group would receive feedback via the traditional post-observation conference. After each weekly session, coach and pre-service teacher were asked to make written notes about the effectiveness of their feedback method. Four weeks of observation and post-observation conferences were planned. Table 5-2 summarizes the steps taken. Table 5-2: Protocol for the control and experimental group according to the six steps approach by Coninx et al. (2012) Protocol for this study
Step 1 : Taking care of two prerequisites (willingness and approval) Step 2: Conference prior to the field experience 1. Explanation of the procedures and roles of coach and pre-service teacher 2 Explanation of the targeted behavior on classroom management, activating instruction and differentiation towards different pupils 3 Instruction in the technical aspects of using a BIE device 4 Modeling good teacher behavior by coach 5 Practice time using a BIE device Step 3 : Discussing of measurements of performance pre-service teachers Pretest: Administering the QTI by pupils Step 4: intervention (four sessions in four consecutive weeks) with pre-observation conference, observation by a coach of an actual lesson sitting in the back of a class room, post-observation conference in which targeted performance is discussed Step 5 : Fading the use of the BIE Step 6: Evaluation of performance of pre-service teacher Posttest: Administering the QTI by pupils
80
Control group (asynchronous coaching) N=15 N = 15
Experimental group (synchronous coaching) N=15 N = 15
N = 15
N = 15
N = 15
N = 15 + compose ‘codes’ (= in advance structured feedback) N=15 N = 15 + modeling usage of BIE by coach N = 15 N = 15 N=282 N = 15 + synchronous coaching
not applicable N = 15 not applicable N = 15 N = 334 N = 15
not applicable N = 15 N = 305 30 pairs of pre-service teachers and their coaches
N = 15 N = 15 N=276 30 pairs of pre-service teachers and their coaches
Observed behavioral change of pre-service teachers 5.4.4
Analyses
In order to estimate the reliability and validity of the QTI, two statistical computations were performed. Cronbach’s alpha coefficient was calculated to test internal consistency of each subscale. Also, correlations between the subscales were calculated. From the circumflex nature of the QTI we expected strong negative correlations between the opposite subscales and positive relations between the adjacent scales. In order to determine a difference between the behavioral change of the experimental (synchronous coaching) and the control group (asynchronous coaching) we used an independent -samples t-test on each scale of the QTI. That is: Behavioral change = QTI (Time 2) - QTI (Time 1)
5.5 Results First, the statistical data for the reliability of QTI are presented in Table 5-3. The alpha reliability coefficients for the different scales of QTI using the class as a unit of analysis are sufficient (Field, 2009). For the QTI pre-test the reliability coefficients range from .69 for the Pupil responsibility scale to .86 for the Understanding scale. Reliability of all the scales was sufficient (Field, 2009): Leadership (.84), Helping/Friendly (.85), Understanding (.86), Pupil Responsibility (.69), Uncertain (.80), Dissatisfied (.81), Admonishing (.86) and Strict (.71). For the QTI post-test the reliability coefficients range from 0.68 for the Pupil Resposability scale to 0.88 for the Understanding scale. Reliability of all the scales is sufficient (Field, 2009): Leadership (.87), Helping/Friendly (.87), Understanding (088), Pupil Responsibility (.68), Uncertain (.85), Dissatisfied (.80), Admonishing (.84) and Strict (.71). The QTI has demonstrated high reliabilities in various countries (e.g., den Brok, Brekelmans, Levy & Wubbels, 2002; Lee & Fraser, 2002; Wubbels & Levy, 1993) as shown in Table 5-3. Table 5-3 Internal consistency (alpha reliability) for QTI scales for pupils measured in four studies and this study Scale QTI measured in five different studies
(a)
(b)
(c)
(d)
DC Leadership CD Helping/friendly CS Understanding SC Pupil responsibility/freedom SO Uncertain OS Dissatisfied OD Admonishing DO Strict
.83 .90 .90 .74 .79 .86 .81 .78
.80 .88 .88 .76 .79 .83 .84 .80
.86 .88 .88 .69 .78 .83 .84 .68
.79 .73 .68 .57 .68 .68 .66 .51
This study (pre-test) .84 .85 .86 .69 .80 .81 .86 .71
Sample size
1105
1606
489
705
616
This study (post-test) .87 .87 .88 .68 .85 .80 .84 .71 581
(a) (Wubbels, Brekelmans & Hooymayers, 1991, (b) (Wubbels & Levy, 1991, 1993) (c) ( Fisher & Rickards (1998), (d) (Gupta & Fisher, 2011).
81
Chapter 5 Second, a further analysis was also carried out to explore the inter-scale correlations between the different scales of the QTI. The QTI is based on a circumflex model in which the scales are arranged to form a circular pattern of the eight dimensions of interpersonal behavior and they are expected to be correlated in a specific way. The Model of Interpersonal Behavior (Wubbels & Levy, 1993) predicts that the correlations between two adjacent scales are highest, but correlations gradually decrease as the scales move further apart until opposite scales are negatively correlated. This pattern is reflected in Table 5-4. Where the results of the inter-scale correlations from the study generally reflect the circumflex nature of the QTI pretest and thus further confirm the validity of QTI to be used in this classroom setting. Table 5-4 Inter Scale Correlations for the Questionnaire on Teacher Interaction (QTI) pre-test Scale
Lea DC
Leadership [DC] Helping/ Friendly [CD] Understanding [CS] Pupil Responsib/Freedom [SC] Uncertain [SO] Dissatisfied [OS] Admonishing [OD] Strict [DO]
HFr CD .846**
Und CS .874** .967**
SRf SC .013 .289 .244
Unc SO -.872** -.815** -.825** .174
Dis OS -.805** -.896** -.886** -.264 .815**
Adm OD -.682** -.836** -.837** - .470** .617** .919**
Str DO -.464** -681** -.652** -.699** .384* .738** .869**
** Correlation is significant at the 0.01 level (2-tailed) * Correlation is significant at the 0.05 level (2-tailed)
The same pattern of circumflex (correlations between two adjacent scales are highest, and opposite scales are negatively correlated) is reflected in Table 5-5 for QTI post-test. Table 5-5 Inter Scale Correlations for the Questionnaire on Teacher Interaction (QTI) post-test Scale Leadership [DC] Helping/ Friendly [CD] Understanding [CS] Pupil Responsib/Freedom [SC] Uncertain [SO] Dissatisfied [OS] Admonishing [OD] Strict [DO]
Lea DC
HFr CD .903**
Und CS .866** .945**
SRf SC -.237 .069 .024
Unc SO -.746** -.660** -.691** .256
Dis OS -.736** -.769** -.769** .086 .823**
Adm OD -.618** -.693** -.691** -.253 .701** .899**
Str DO -.266 -.463** -.412* -.618** .389* .663** .834**
** Correlation is significant at the 0.01 level (2-tailed) * Correlation is significant at the 0.05 level (2-tailed)
Third, we conducted an independent-samples t-test on two dimensions and eight subscales of the QTI, as shown in Table 5-6 and Table 5-7. An indepent t-test was performed on the QTI scores. The difference in ‘growth’ between the experimental en control groups was compared. Growth is the difference between the scores on the pre- and post-test.
82
Observed behavioral change of pre-service teachers Table 5-6 Mean and standard deviation for the two dimension of the QTI Synchronous coaching (N=30)
Asynchronous coaching (N=30)
Scale of the QTI
Proximity dimension (Cooperation, C – Opposition) Influence dimension (Dominance, D - Submission, S)
M
SD
M
SD
.13 .10
.30 .21
-.21 -.02
.34 .18
For the ‘influence dimension’ an independent-samples t-test indicated no significant difference (t (30) = 1.66, p = .11, two-tailed) between the control group (M = .10, SD = .21), and the experimental group (M=-.02, SD=.18). However, for the ‘proximity dimension’ an independent-samples t-test indicated a significant difference (t (30) = 2.91, p = .007, two-tailed) between the control group (M = -.21, SD =.34), and the experimental group (M=.13, SD=.30). The magnitude of the difference in the means was large (Cohen`s d= 1.10). Pre-service teachers who received synchronous coaching were seen by their pupils more cooperative than teachers who did received asynchronous coaching. If we look closer to the eight subscales we can see that the magnitude of behavioral change for the experimental group can be explained by three subscales namely the Leadership [DC], Helping/ Friendly [CD] and the Understanding [CS] subscale. Table 5-7 Mean and standard deviation for each sub scale of the QTI Synchronous coaching (N=30)
Asynchronous coaching (N=30)
Sub Scale
Leadership [DC] * Helping/ Friendly [CD] * Understanding [CS] * Pupil Responsib/Freedom [SC] Uncertain [SO] Dissatisfied [OS] Admonishing [OD] Strict [DO]
M
SD
M
SD
.09 .07 .06 .00 .01 .00 .01 .02
.13 .09 .10 .05 .05 .05 .08 .06
-.02 -.08 -.09 .01 .02 .02 .02 .02
.07 .13 .12 .06 .07 .07 .06 .08
For the subscale ‘Leadership’ an independent-samples t-test indicated a significant difference (t (30) = 2.92, p = .007, two-tailed) between the control group (M = -.02, SD = .07), and the experimental group (M = .09, SD = .13). The magnitude of the difference in the means was large (Cohen`s d = 1.09). Pre-service teachers exposed significant more ‘leadership’ in the experimental group (synchronous coaching) as reported by pupils. For subscale ‘Helping’ an independent-samples t-test indicated a significant difference (t (30) = 3.67, p = .001, two-tailed) between the control group (M = -.08, SD = .13), and the experimental group (M = .07, SD =. 09). The magnitude of the difference in the means was large (Cohen`s d = 1.39). Pre-service teachers who received synchronous coaching (experimental group) exposed significant more ‘helping’ behavior than the pre-service teachers of the asynchronous coaching group.
83
Chapter 5
For subscale ‘Understanding’ an independent-samples t-test indicated a significant difference (t (30) = 3.72, p = .001, two-tailed) between the control group (M = -.09, SD = .12), and the experimental group (M = .06, SD = .10). The magnitude of the difference in the means was large (Cohen`s d = 1.41). Pre-service teachers of the synchronous coaching group were observed more understandingly by pupils than the pre-service teachers of the asynchronous coaching group.
5.6 Conclusion and Discussion Our results confirmed prior findings regarding the benefits of synchronous coaching to alter teaching behavioral changes in the desired direction as perceived by the pupils. Pre-service teachers who received synchronous coaching, scored significantly higher on the Proximity dimension (Cooperation, C - Opposition, O) than the pre-service teachers of the control group (asynchronous coaching). So, the former pre-service teachers became more cooperative and had more ascendancy in their class room. Pre-service teachers of the synchronous coaching group were seen as more friendly and understandingly. It is, therefore, expected that their pupils will show higher affective and cognitive outcomes (Wubbels et.al., 1991). These findings are in line with other research on synchronous coaching. Researchers found that synchronous coaching had a positive impact on pre-service teacher behavior (Bowles & Nelson, 1976; Giebelhaus and Cruz, 1994; Rock et al., 2009; Scheeler & Lee, 2002; Scheeler et al., 2006; Scheeler, Bruno & Grubb, 2009; Silverman and Kimmel, 1972; Van der Mars, 1988). For example, in the study of Rock and colleagues (2009a) the positive impact on teacher behavior was seen in statistically significant increases in teachers’ use of instructional activities and teachers’ praise. As teacher’s behavior is one of the most important factors affecting pupil’s performance, it is essential that researchers provide empirical evidence supporting those changes in teacher behavior due to effective interventions such as synchronous coaching with a BIE device (cf., Rock et al., 2009b). By collecting data of changes that occurred in the perceptions of pupils, we further validated that what teachers do in the class room is noted by pupils and really does matter. With respect to the dimensions leadership, helping/friendly and understanding significant differences are noted and effect sizes are large, indicating that synchronous coaching is effective with respect to these dimensions. Furthermore, we suggest that our findings regarding the use of synchronous coaching have several important implications for teacher education research and practice. This study underscores the fact that synchronous coaching is a practical tool with which to strengthen the teaching and learning process of pre-service teachers. The practical value is that it may help schools, school leaders and teacher education programs to view synchronous coaching as one element of a more comprehensive treatment intervention. It is apparent that asynchronous coaching does not guarantee effective behavior changes in the classroom, which is the generally accepted goal of pre-service teachers training. One suggestion to improve or to replace pre-service training is to provide more direct in-class synchronous coaching than is frequently offered (Bowles & Nelson, 1976). Perhaps the same strategy can be applied with the preparation of coaches. Schools can adopt similar strategy of training coaches to reliably evaluate targeted teaching skills and implement a coaching protocol. This could be accomplished by developing a series of activities as part of training program which would be required by all coaches. Such training could potentially promote consistency and quality of coaching across coaches (O’Reilly et al., 1994). 84
Observed behavioral change of pre-service teachers
However, there are some limitations regarding this study. First, in the studies of synchronous coaching researchers frequently noted that technical issues did interfere occasionally with successful BIE use (Giebelhaus & Cruz, 1994; Herold, Ramirez & Newkirk, 1971; Rock et.al., 2009). Note that we lost eight respondents due to technical glitches. When using BIE device, both pre-service teacher and coach have to be patient, willing to engage in on-the-spot troubleshooting, and have access to some type of on-going technology support (Rock et al., 2009a). Second, although we randomly assigned the pre-service teachers and their coaches to two different coaching modes (synchronous coaching versus asynchronous coaching) we selected a limited number of pairs (N=30). Although this number is low in quantitative research, only a very few studies reached the same number of participants as in our study. Nevertheless, further research can expand the number of respondents. Third, we included no follow-up observations to document pre-service teachers continued behavior change. As a result, we do not know whether the improvements were maintained over time. Future researchers may increase the length of the experiment to allow the assessment of maintenance, and generality of the findings to more applied settings. Fourth, in the present research, effects of synchronous coaching were investigated via pupils’ perceptions of their pre-service teacher and not their actual learning. The use of self-reports data (i.e. the pre-service teachers administrated QTI) was avoided to decrease the issue of overestimation (Fisher et al., 1995). Therefore, we included the perceptions of pupils. In learning environment research, pupils’ perceptions are typically analysed because these perceptions are assumed to be more similar to the observations and judgments of experts (i.e. researchers or external observers) than the perception of pre-service teachers themselves (Fisher et.al., 1995). But this means that pupils’ learning was not actually assessed. In teacher training research, there are effects to consider in addition to changes in teachers’ behavior such as an increase in pupil’s cognitive and affective achievements. Future research on providing feedback to pre-service teachers should also include assessment of pupil’s learning to determine if changes in teacher behavior affects changes in student achievement (fourth level of evaluation of Kirkpatrick, 2002). Lacking an effective means of providing intervention and feedback during the on-going teaching process, coaches have relied on the delayed and frequently awkward retrospective approach of post observation discussions (Giebelhaus, 1994). This study was initiated to address the need to provide a more systematic method of offering corrective, immediate feedback. If the goal of pre-service training program is to train teachers to behave more productively in their class room then (1) their behavior must be measured in the class room to determine if the desired changes do in fact occur, (2) a variety of techniques, including ones as affective as synchronous coaching, must be used to shape the behaviors that are set as the criteria for the teachers to obtain.
References Billingsley, B.S., Carlson, E., & Klein, S. (2004). The working conditions and induction support of early special career special educators. Exceptional Children, 70(3), 333-347. Boe, E.E., Cook, L.H., & Sunderland, R.J. (2008). Teacher turnover: Examining exit attrition, teaching area transfer, and school migration. Exceptional Children, 75(1), 7-31.
85
Chapter 5 Bowles, E.P., & Nelson, R.O. (1976). Training teachers as mediators: Efficacy of a workshop versus the bug-in-ear technique. Journal of School Psychology, 14(1), 15-25. Brekelmans, M., Wubbels, Th., & Brok, P. den (2002). Teacher experience and theteacherstudent relationship in the classroom environment. In S. C. Goh & M. S. Khine (Eds.), Studies in educational learning environments: an international perspective (pp.73-99). Singapore: World Scientific Brok, P.J. den, Brekelmans, J.M.G., Levy, J. & Wubbels, Th. (2002). Diagnosing andimproving the quality of teachers' interpersonal behaviour. International journal of educational management, 16(4), 176-184. Brown, S.W. (2005). Emily and Rebecca: A tale of two teachers. Teaching and Teacher Education, 21(6), 637-648. Brownell, M.T., Hirsch, E., & Seo, S. (2004). Meeting the demand for highly qualified special education teachers during severe shortages: What should policymakers consider? Journal of Special Education, 38(1), 56-61. Clutterbuck, D. (2004). Everyone needs a mentor: fostering talent at work. London, CIPD. Coninx, N.S., Kreijns, C.J., & Jochems, W.M.G. (2013). The use of keywords for delivering immediate performance feedback on teacher competence development. European Journal of Teacher Education, 36(2), 164–182. Darling-Hammond, L. (2000). How teacher education matters. Journal of Teacher Education, 51(3), 166-73. Darling-Hammond, L., & Bransford, J. (with LePage, P., Hammerness, K., & Duffy, H.). (2005). Preparing teachers for a changing world: What teachers should learn and be able to do. San Francisco: Jossey-Bass. Farrell, A.C., & Chandler, D. (2008). Cooperating teachers’ impressions of the Whisper in my ear (WIME) and traditional communication feedback methods for physical education pre service teachers. Education and human development, 2(1), 1-9. Feiman-Nemser, S. (2001). From preparation to practice: Designing a continuum to strengthen en sustain teaching. Teacher College Record, 103, 1013-1055. Field, A. P. (2009). Discovering statistics using SPSS. London, England : SAGE. Fisher D., Rickards, T. (1998). Associations between Teacher-Student Interpersonal Behaviour and Student Attitude to Mathematics. Mathematics Education Research Journal, 10(1), 3-15. Fry, J.M., & Hin, M.K.T. (2006). Peer coaching with interactive wireless technology between student teachers: satisfaction with role and communication. Interactive learning environments, 14(3), 193-204. Giebelhaus, C. R., & Cruz, J. (1992). The third ear mechanical device: A supervision alternative. Journal of Early Childhood Teacher Education, 42(13), 8-12. Giebelhaus, C.R. (1994). The bug-in-the-ear device: An alternative student teaching supervision strategy. Paper presented at the association of teacher educators’ annual conference. Atlanta, GA. Giebelhaus, C. R., & Cruz, J. (1994). The mechanical third ear device: An alternative to traditional student teaching supervision strategies. Journal of Teacher Education, 45, 365-373 Giebelhaus, C. R. & Bowman, C. L. (2002). Teaching mentors: Is it worth the effort? The Journal of Educational Research, 95(4), 246-254. Goodman, J.I., & Duffy, M.L. (2007). Using BUGS to increase student participation. Teaching exceptional children plus, 3(4), retrieved from: http://scholarship.bc.edu/education /tecplus/vol3/iss4/art3. Greer, R.D. (2002) The learning unit: a natural fracture of teaching. Designing Teaching Strategies, 17-39.
86
Observed behavioral change of pre-service teachers Gupta, A., & Fisher, D. (2011). Teacher-student interactions in a technology-supported science classroom environment in relation to selected learner outcomes: an Indian studymier. Journal of Educational Studies, Trend & Practices, 1(1), 34-45. Hashweh, M.Z. (2003). Teacher accommodative change. Teaching and Teacher Education, 19, 421-434. Helms-Lorenz, M., Slof, B., Vermue, C. E., & Canrinus, E.T. (2012a). Beginning teachers’ self-efficacy and stress and the supposed effects of induction arrangements. Educational Studies, 38(2), 189-207. Herold, P., Ramirez, M., & Newkirk, J. (1971). A portable radio communication system for teacher education. Educational Technology, 11, 30-32. Houtveen, T., Versloot, B., & Groenen, I. (2006). De begeleiding van beginnende leraren. [Coaching of beginning teachers]. Utrecht, Sectiebestuur Onderwijsarbeidsmarkt (SBO). Houtveen, T., & Van de Grift, W.J.C.M. (2012). Improving reading achievements of struggling learners. School Effectiveness and School Improvement, 23(1) 71-93. Kahan, D. (2002). The effects of a bug-in-the-ear device on intra lesson communication between a student teacher and a cooperating teacher. Journal of Teaching in Physical Education, 22(1), 86-104. Kersaint, G., Lewis, J., Potter, R., & Meisels, G. (2007). Why teachers leave: Factors that influence retention and resignation. Teaching and Teacher Education, 23(6), 775-794. Kirkpatrick, D. & Kirkpatrick, P. (2006). Evaluating Training Programs (3rd ed.). San Francisco, CA: Berrett-Koehler Publishers, Inc. Leary, T. (1957). Interpersonal diagnosis of personality: a functional theory and methodology for personality evaluation, New York: Ronald Press Co. Lee, S.S.U. & Fraser, B.J. (2002, April). High school science classroom learning environments in Korea. Paper presented at the annual meeting of the American Educational Research Association. New Orleans, LA. Neukrug, E. (1991). Computer-assisted live supervision in counselor skills training. Counselor Education and supervision, 31, 132-138. O’Reilly, M. F., Renzaglia, A., & Lee, S. (1994). An analysis of acquisition, generalization and maintenance of systematic instruction competencies by pre-service teachers using behavioral supervision techniques. Education and Training in Mental Retardation and Developmental Disabilities, 29, 22-33. Price, A. T., Martella, R. C., & Marchand-Martella, N. (2002). A comparison of immediate feedback delivered via an FM headset versus delayed feedback on the inappropriate verbalizations of a student with ADHD. Education and Treatment of Children, 25, 159171. Rathel, J.M., Drasgow, E. & Christle, C.C. (2008). Effects of supervisor performance feedback on increasing pre-service teachers’ positive communication behavior with students with emotional and behavioral disorders. Journal of Emotional and behavioral disorders, 16(2), 67-77. Rock, M.L., Gregg, M., Thead, B.K., Acker, S.E., Gable, R.A., & Zigmond, N.P. (2009a). Can You Hear Me Now? Evaluation of an online wireless technology to provide realtime feedback to special education teachers-in-training. Teacher Education and Special Education, 32(1), 64-82. Rock, M.L., Gregg, M., Howard, P.W., Ploessl, D.M., Maughn, S., Gable, R.A., & Zigmond, N.P. (2009b). See me, hear me, coach me. Virtual bug-in-ear technology brings immediacy to professional development. National staff development council, 30(3), 2432.
87
Chapter 5 Rushton, S. P. (2003). Two pre-service teachers’ growth in self-efficacy while teaching in an inner-city school. The Urban Review, 35(3), 167-189. Scheeler, M.C., & Lee, D.L. (2002). Using technology to deliver immediate corrective feedback to preservice teachers. Journal of behavioral education, 11(4). 231-241. Scheeler, M.C., Ruhl, K.L., & McAfee, J.K. (2004). Providing performance feedback to teachers: a review. Teacher education and special education, 27(3), 396-407. Scheeler, M.C., McAfee, J.K., Ruhl, K.L., & Lee, D.L. (2006). Effects of corrective feedback delivered via wireless technology on preservice teacher performance and student behavior. Teacher education and special education, 29(1), 12-25. Scheeler, M.C., Macluckie, M., & Albright, K. (2008). Effects of immediate feedback delivered by peer tutors on the oral presentation skills of adolescents with learning disabilities. Remedial and special education, 11, 1- 10. Scheeler, M. C., Bruno, K. & Grubb, E. (2009). Generalizing Teaching Techniques from university to K-12 classrooms: teaching pre-service to use what they learn. Journal of Behavioral Education, 18, 189-210. Silverman, S. & Kimmel, J. (1972). The influence of immediate feedback on the behavior of teacher-in-training (Final Report No. ED068463). Gainesville, FL : University of Florida, and the National Center for Educational Research and Development. Smith, T. M., & Ingersoll, R. M. (2004). What are the effects of induction and mentoring on beginning teacher turnover? American Educational Research Journal, 41(3), 681-714. Smith, D. L. & Thompson, S. (2005). Creating highly qualified teachers for urban schools. Professional Educator, 27, 73-88. Stroot, S., Keil, V., Stedman, P., Lohr, L., Faust, R., Schincariol-Randall, L., Sullivan, A., Czerniak, G., Kuchcinski, J., Orel, N., & Richter M. (1998). Peer assistance and review guidebook. Columbus, OH, Ohoi Departement of Education. Swafford, J. (1998) Teachers Supporting Teachers through Peer Coaching. Support for Learning, 13 (2), 54-58. Van der Mars, H. (1988). The effects of audio-cueing on selected teaching behaviors of an experienced elementary physical education specialist. Journal of Teaching in Physical Education, 8(1), 64-72. Veenman, S. (1984). Perceived problems of beginning teachers. Review of Educational Research, 54 (2), 143-178. Watzlawick, P., Beavin, J. H., & Jackson, D. (1967). The pragmatics of human communication. New York: Norton. West, E., & Jones, P., (2007). A framework for planning technology used in teacher education programs that serve rural communities. Rural Special Education Quarterly, 26(4), 3-15. White, J., & Gordon, J. (2000). There’s a bug in my ear: Value adding trough authentic interview experience. Contemporary Issues in Early Childhood, 2(1), 287-298 Wong, H.K., Britton, T., & Gasner, T. (2004). What the world can teach us about improving new teacher induction. Phi Delta Kappan, 86(5), 379-384. Woullard, R. & Coats, L. T. (2004). The community college role in preparing future teachers: The impact of a mentoring program for pre-service teachers. Community College Journal of Research and Practice, 298, 609-624. Wubbels, T., Brekelmans, M., & Hooymayers, H. (1991). Interpersonal teacher behaviour in the classroom. In B.J. Fraser & H.J. Walberg (Eds.). Educational environments: Evaluation, antecedents and consequences (pp. 141-160). Oxford, England: Pergamon Press. Wubbels, T., Creton, H.& Holvast, A. J. (1988). Undesirable classroom situations. Interchange, 19(2), 25-40.
88
Observed behavioral change of pre-service teachers Wubbels, T., Creton, H. A., & Hooymayers, H. P. (1992). Review of research on teacher communication styles with use of the Leary model. Journal of Classroom Interaction, 27,1-12. Wubbels, T., & Levy, J. (1991). A comparison of interpersonal behaviour of Dutch andAmerican teachers. International Journal of Intercultural Relations, 15, 1-18. Wubbels, T., & Levy, J. E. (1993). Do you know what you look like? Interpersonal Relationships in Education (1st. ed.). London: The Falmer Press.
89
Chapter 5
90
Chapter 6 Conclusion and discussion Abstract: Teachers are identified as an important factor influencing the quality of education in schools. The quality of the development of these teachers depends on the support that is given in each of the three phases: teacher training (training of pre-service teachers before entering the classroom as a fully responsible teacher), induction (a process of coaching and support during the first teaching practice in a actual classroom of a particular school) and continuing professional development (an in-service process for practicing teachers). The teaching profession has alarmingly high rates of dropouts during the induction phase. Therefore, it is important to optimalize teacher training programs. This research aimed at developing and evaluating a coachings program, as a part of a teacher traininge program, namely synchronous coaching. Different factors contributing to the effectiveness of synchronous coaching using BIE device were first investigated by deriving a six step approach. Next, the innovative use of keywords was measured. Further, in depth investigations were conducted to measure the effectiveness of synchronous coaching on the level of perceived satisfaction and on the level of behavioral change. Subsequently, strengths and limitations of the research are discussed, followed by suggestions for future research. Finally, theoretical and practical implications of this research will be presented.
6.1 Background to the study New teachers indicated a number of reasons for leaving the profession after their first year (Boe, Cook & Sunderland, 2008; Kersaint, Lewis, Potter & Meisels, 2007; Smith & Ingersoll, 2004). These reasons are related to the support they (don’t) get during their teacher training phase. Now there is more emphasis placed on the school as a learning environment where teachers’ professional learning can be supported (Hennissen, Crasborn, Brouwer, Korthagen & Bergen, 2008). In order to optimize teacher professional development more systematically and empirically validated research is needed (Hasweh, 2003). Effective and efficient teacher teacher training programs are the ultimate goal (Wade, 2010). The perspective used in this research was the micro perspective, in which the individual preservice teacher and his work in front of the class were taken as viewpoint. In this perspective the improvement of teacher training was considered a central element of a solution (Veenman, 1984). The teacher training phase was considered as an important phase in teacher development within the first years of employment. The scope of this study was limited to the coaching of pre-service teachers and aimed at preparing them better for their first years in front of the class. The focus of this research has been on the training program, especially on the effectiveness and efficiency of this program. The aim of this research was to measure the effectiveness of synchronous coaching using a BIE device. So, the general research question was: “To what degree is synchronous coaching using BIE device effective in terms of behavioral change in the proposed desired direction (= becoming more cooperative and having more ascendancy in the pre-service’s class room)?” 91
Chapter 6 In order to address the general research question, four more specific research questions were derived: a) Which systematic approach is suitable for immediate performance feedback using BIE device of pre-service teachers? (Chapter 2)? b) Does this systematic approach reduce cognitive load and ambiguity (Chapter 3)? c) To what degree is the six steps (systematic) approach effective for perceived satisfaction of pre-service teachers (Chapter 4)? d) To what degree is the six steps (systematic) approach of synchronous coaching effective for behavioral improvement of pre-service teachers (Chapter 5)? Different factors contributing to the effectiveness of synchronous coaching using a BIE device were first investigated by deriving a six step approach, described in Chapter 2. Next, the innovative use of keywords was measured, presented in Chapter 3. An in depth investigations was conducted to measure the effectiveness of synchronous coaching on the level of Reaction, i.e. perceived satisfaction which has been reported in Chapter 4 and on the level of Behavior Change as described in Chapter 5. These research questions were addressed as 4 main themes and will be summarized in this chapter, sections 6.2.1 to 6.2.4. Subsequently, strengths and limitations of the research are discussed (section 6.3), followed by suggestions for future research (section 6.4). Finally, theoretical and practical implications of this research (section 6.5) will be presented.
6.2 Brief overview of the research project 6.2.1
Synchronous coaching
In answering the first research question, the following definition of synchronous coaching was derived: In education, synchronous coaching is a method for giving immediate performance feedback with certain characteristics (specific, corrective, encouraging) by using BIE on targeted performance. These feedback messages are discussed in advance. Most commonly a cooperating teacher is sitting in the back of a classroom and sends the feedback to a preservice teacher. A six steps approach specifying the successive actions in combination with a training program for sending immediate performance feedback is preferable. We conducted a qualitative, literature search to develop a definition of synchronous coaching. A total of 29 descriptions were extracted from 79 articles. These were systematically analyzed for common characteristics. A six steps approach was developed to summarize the current understanding of synchronous coaching: 1) Taking care of two prerequisites, a) Both coach and pre-service teacher must be willing to participate, b) Getting parental permission if pupils in the class room are being filmed or interviewed for research purposes, 2) Implementing a training program for using a BIE device, 3) Discussing performance of the pre-service teacher, 4) Intervention: the actual usage of the BIE device in the classroom: preferably four times (Farrell & Chandler, 2008; Fry & Hin, 2006; Kahan, 2002; O’Reilly, Renzaglia & Lee, 92
Conclusion and discussion 1994; Price, Martella, Marchand-Martella, 2002; Rock, Gregg, Thead, Acker, Gable & Zigmond, 2009a) each lesson ending in a reflection. In this reflection the next targeted behavioral improvement and next codes should be discussed, 5) Fading the usage of BIE (keeping BIE in the ear but not giving immediate performance feedback by coach and at the end not using the BIE device at all), and 6) Evaluation of performance of pre-service teacher.
6.2.2
Cognitive load and ambiguity
After defining a six steps (systematic) approach, another element of the BIE device, namely pre-structured keywords, needed to be more elaborated. Due to immediateness of feedback pre-service teachers can suffer from cognitive overload, caused by the combination of information they receive from the class and the feedback. We proposed a set of standardized keywords through which this immediate feedback can be delivered; each keyword acted as a summary for the feedback message. The construction and the selection of the keywords was aimed to the reduction of message ambiguity, while at the same time a low level of cognitive load on the pre-service teacher should be maintained. An in vivo pilot-study with forty respondents (pre-service teachers and their coaches) supported our hypothesis that usage of such set of standardized keywords will mitigate the levels of ambiguity and cognitive load. The results of this study indicated that the use of structured keywords for delivering immediate performance feedback is more beneficial than the ad hoc mode of delivering performance feedback. Beneficial was defined in terms of producing less cognitive load and reducing ambiguity for pre-service teachers. For ambiguity, this seems logical because in the structured mode coach and pre-service teacher always have to select and discuss in advance the keywords they are going to use. Besides, the keywords were well known to both the pre-service teacher and the coach. This diminished the risk of ambiguity. The reasoning related to cognitive load is slightly different. The increased amount of cognitive processing required performing a task while receiving, interpreting and responding to immediate feedback may impose high extraneous cognitive load on pre-service teachers (Walsh, Ling, Wang & Carnaham, 2009). By structuring the messages and only using the selected keywords known to the pre-service teacher it was expected that cognitive load (i.e. extraneous load) would be small, resulting in lower reported levels of cognitive load.
Non verbal behavior
Class room management
Table 6-1: Feedback messages (or keywords) for synchronous coaching (Marzano, Pickering & Pollock, 2001; Marzano, Pickering & Pollock, 2009) Feedback message English Basic attitude (to stand, walk or sit) Shake head Nod Gestures Move Territory (getting into the territory of pupils) Body orientation Touch Eye contact Fixate Intonation
References (d), (e), (f) (d), (e), (f) (d), (e), (f), (b), (d), (e), (f), (d), (e), (f) (d), (e), (d), (e), (f), (d), (e), (f), (b), (c), (d), (e), (f) (d), (e) (d), (e), (f)
93
Feedback on Process Encourage Disapprove
pedagogical action (relation pupil - teacher)
Content
Instruction
didactic instruction
Feedback on Result
Chapter 6 Feedback message English Voice volume Pause Show (work of pupils) Laugh Disapprove Indicate what is correct Indicate what is correct is and to tell why Give a suggestion Give a suggestion and to tell why Indicate a good solution Indicate a good solution and to tell why Indicate the standard Clarify the problem Give hints Show Explain the term Explain the structure Let a pupil exactly define the problem Ask a pupil what he/she has already done Ask a pupil for his knowledge Stimulate a pupil his thinking process Ask if a pupil understands it Instruction who Instruction what Instruction how Instruction time periode Instruction why Pronouncement Calculation mistake Writing mistake
References (d), (e), (f), (b), (c), (d), (e), (f) (d), (e), (f), (d), (e), (f), (d), (e), (f), (h) (b), (d), (e), (f), (g), (h) (b), (d), (e), (f), (g),(h) (a), (b), (d), (e), (g), (f), (h) (b), (d), (e), (g), (f), (h) (b), (d), (e), (g), (f), (h) (a), (b), (d), (e), (g), (f), (h) (a), (b), (d), (e), (f), (h) (a), (b), (d), (e), (f), (h) (a), (d), (e), (f), (h) (a), (d), (e), (f), (h) (a), (d), (e), (f), (h) (a), (d), (e), (f), (h) (a), (b), (d), (f), (h) (b), (c), (d), (e), (f), (h) (a), (b), (c), (d), (e), (f), (h) (b), (c), (d), (e), (f), (h) (b), (c), (d), (e), (f), (h) (c), (d), (e), (g), (f) (c), (d), (e), (g), (f) (c), (d), (e), (g), (f) (c), (d), (e), (g), (f) (c), (d), (e), (g), (f) (d), (e) (d), (e) (d), (e), (f)
Next working form Select ad random pupils for answering Summarise Give every pupil attention Chalkboard neater Encouraging message for a pupil Non verbal encouraging for a pupil
(b) (b) (a), f) (b), (f) (a) (b), (d), (e), (f), (g), (h) (b), (d), (e), (f), (g), (h)
Throw out a pupil Punish a pupil Give detention to a pupil Move a pupil
(d), (e), (g), (h) (d), (e), (g), (h) (d), (e), (g), (h) (d), (e), (g), (h)
Warn a pupil (d), (e), (g), (h) Disapprove a pupil with a I message (d), (e), (g), (h) Name a pupil and to continue the lesson (d), (e), (g), (h) Look at a pupil and to stop the lesson (d), (e), (g), (h) Look at a pupil and to continue the lesson (d), (e), (g), (h) Ignore a pupil (d), (e), (g), (h) (a) (Giebelhaus & Cruz, 1994); (b) (Goodman, Brady, Duffy, Scott & Pollard, 2008); (c) (Goodman & Duffy, 2007); (d) (Rathel, Drasgow & Cristle , 2008); (e) (Rock, Gregg, Howard, Ploessl, Maughn, Gable & Zigmond, 2009b); (f) (Scheeler, Bruno & Grubb, 2009); (g) (Hunt, 1980); (h) (Bowles & Nelson, 1976)
94
Conclusion and discussion 6.2.3
Perceived satisfaction of pre-service teachers
Now that synchronous coaching was defined, a study was conducted to measure the effects of synchronous coaching. Research on the effectiveness of a coaching program can be performed on four levels according Kirkpatrick & Kirkpatrick (2006): 1) level of reaction or perceived satisfaction, 2) level of knowledge, skills and attitudes, 3) level of behavior (a change in behavior), and finally 4) level of results (effects on pupils). In this study we have focussed on the first level, the level of reaction (Kirkpatrick & Kirkpatrick, 2006), namely perceived satisfaction. To this end, a literature search was conducted on perceived satisfaction of synchronous coaching, combined with an in vivo study with eighty respondents (pre-service teachers and their coaches). The in vivo study supported the literature findings that according to preservice teachers synchronous coaching is more supportive, is providing more useful feedback, and is more encouraging than asynchronous ways of coaching. The synchronous coaching group showed a significantly greater satisfaction for three functions, namely supportiveness, giving feedback and encouragement, than the asynchronous coaching group. The synchronous coaching group also outperformed the asynchronous group on the other two functions, discussing suggestions and discussing the effects on pupils, but these differences were not statistically significant. Synchronous coaching with the use of BIE-device during pre-service teachers’ field experiences appeared to be a valuable tool for coaches to provide immediate performance feedback when it comes to the satisfaction of the pre-service teachers. But have these teachers changed their teaching behavior? Our next studie therefore should answer the question whether synchronous coaching will lead to more learning activities and eventually more effective teaching behavior. 6.2.4
Behavioral improvement of pre-service teachers
The last part of our research was to measure the effects of synchronous coaching on behavioral change of pre-service teachers (the third level of evaluation of a training program according to Kirkpatrick & Kirkpatrick, 2006). In the application of the Leary model (1957), teacher behavior is mapped on a Proximity dimension (Cooperation, C - Opposition, O) and an influence dimension (Dominance, D - Submission, S). The Proximity dimension can be depicted on a continuum that has cooperative behavior at one end and oppositional behavior at the other; the Influence dimension on a continuum with dominant behavior at one end and submissive behavior at the other Regarding students’ cognitive outcomes, the more teachers demonstrated cooperative and ascendant behavior, the higher the cognitive outcomes scores were (Wubbels, Brekelsmans & Hooymayers, 1991). So, in this study effective teaching behavior was defined in terms of getting higher on the Proximity dimension (C - O) and the Influence dimension (D - S). The hypothesis tested is that with immediate feedback (synchronous coaching) pre-service teachers will scored higher on the Proximity dimension and the Influence dimension than in the asynchronous way of coaching (control group). To this end, a literature search was conducted on behavioral change of pre-service teachers who received synchronous coaching. An in vivo study with 1197 pupils and a total of 80 pre-
95
Chapter 6 service teachers and their coaches (40 pairs) supported the literature findings that synchronous coaching is more effective in positively changing the behavior of pre-service teachers than is asynchronous coaching. In particular we found that pre-service teachers became more cooperative and showed more ascendancy in their classroom than pre-service teachers who received asynchronous coaching. Overall, Brekelmans, Wubbels, and den Brok (2002) have indicated that teacher interpersonal behavior is an important aspect of the learning environment and that it is strongly related to student outcomes. Given that teacher’s effectiveness is one of the most important factors affecting pupil’s performance, it is essential that researchers collect outcomes that document those changes in teacher behavior (Rock et al., 2009a). Pre-service teachers receiving synchronous coaching, became more cooperative and had more ascendancy in their classroom. It was expected that their pupils will show higher affective and cognitive learning outcomes (Wubbels, et al., 1991). These findings were in line with other studies on synchronous coaching. Researchers found that synchronous coaching had a positive impact on pre-service teacher behavior (Bowles & Nelson, 1976; Giebelhaus & Cruz, 1994; Rock et al., 2009b; Scheeler & Lee, 2002; Scheeler, McAfee, Ruhl & Lee, 2006, Scheeler et al., 2009; Silverman & Kimmel, 1972; van der Mars, 1988).
6.3 Limitations The results of the study need to be interpreted with caution in some respects, as there are limitations, especially with respect to social validation, generalizability, and definitions. Through the acknowledgment of these limitations, researchers may gain insight regarding the boundaries of this research and the direction for future research. 6.3.1
Social validation
Social validation is defined as the level of social significance of a project, the fidelity of implementation, and the degree of social importance (Horner, Carr, Halle, McGee, Odom & Wolery, 2005). This study presented an intervention aimed at enhancing the support provided to pre-service teachers in a real life classroom. Through an innovative six step approach and the integration of technology, pre-service teachers received immediate feedback and prompting through BIE-device. It provided coaches, the capability to support pre-service teachers adequately. So, as an intervention study the social validation seemed adequate. However, most of the existing research on teacher development is rather uncritical with respect to the outcomes examined. While our research has focused on an important set of outcomes (pre-service teachers’ satisfaction and pre-service teachers’ perceived behavioral change), these do not exhaust the possible outcomes. There are multiple and competing definitions of the goals of pre-service teacher coaching, and hence, also multiple and competing definitions of the “effective” teacher (Ingersoll & Strong, 2011). Definitions of the latter range from teachers who are most able to engage students in higher order and critical inquiry, who are most effective at raising mature citizens, to those who are most sensitive to student diversity, most caring of children, best at promoting students’ social and behavioral development, and most effective at raising student test scores. It is convenient to assume that the “good” teacher is effective at most of the above tasks. But this may not be true. Indeed, coping with multiple and competing tasks has long been recognized as a central challenge for
96
Conclusion and discussion schools and teachers (Bidwell, 1965). Recent research suggests that teachers who are good at promoting some of the goals of education are not necessarily good at promoting other goals (Ingersoll & Strong, 2011). Therefore, more aspects of teacher effectiveness could have been studied. A second limitation of this research is that much attention was placed on short term effects. Some researchers made an effort to measure effects on performance change after six months, other researchers observed a change in performance after only four weeks (van der Mars, 1988; Scheeler, Maclukie & Allbright, 2008). It is a general risk of educational innovations that educational designers develop a blueprint for a powerful learning environment that teacher subsequently do not or cannot fully implement in their teaching. Although synchronous coaching was supportive for teacher educators, this only accounts for the second stage of the pre-service teacher education. Maintenance of performance should be measured systematically over a longer period. A longitudinal approach is therefore preferable. 6.3.2
Generalizability
The participants, for the purpose of the study, were limited to pre-service teacher’s willing to participate in the study. This is because pre-service teachers and their coaches were required to attend the workshop. Because of this preparedness to participate, the experimental group might be a selective group. A side effect might be that - as the literature points out - most of the time these pre-service teachers were more motivated resulting in a positive impact on the measured effects. Another obstacle emerged with respect to the coaches attitudes toward participation in the study. Almost all coaches expressed concern regarding directing and even stearing the preservice teacher. Initially, coaches applying synchronous coaching felt they were prescribing the pre-service teachers what to do. The researcher addressed these concerns of participants and after answering questions regarding the nature of synchronous coaching, coaches appeared to be more receptive to the study and agreed to participate. Also the pre-service teachers’ attitudes towards participation in study could have influenced the results. But, because of the good relationship between coach and pre-service teachers, pre-service teachers felt comfortable participating in the study and welcomed the integration of technology as a means for improving instruction. Additionally, pre-service teachers were informed that participation in the study would give extra information on the performance of pre-service teachers. 6.3.3
Opertionalizations of synchronous coaching
A valuable aspect of this research was the construction of a clear operationalization of how to apply synchronous coaching. This is not only of importance for teacher training purposes, but it also enables comparable studies. In this way, the teaching behaviors expected from the teachers are made explicit and, therefore, better accessible for empirical studies. Besides, as compared to other studies a considerable amount of subjects has been involved in our studies. However, it also implies that other definitions of synchronous coaching might exist, leading to different operationalizations, and eventually to different outcomes. More research is needed in order to get a deeper insight. Besides, hardly any attention is paid to the exact definition of coaching and the similarities or the differences with cognitive activities such as mentoring and training, which frequently discussed in the literature (Engelen, 2002). Besides, themes such as
97
Chapter 6 the hierarchical relationship between the coach and the one being coached (mentoring versus coaching) or the question as to what extent personal development is achieved through synchronous coaching, have not been addressed. We limited ourselves to the question whether synchronous coaching is effective in specific situations.
6.4 Suggestions for future research 6.4.1
Computer-provided immediate specific performance feedback
With the evolution and integration of technology into the field of education coupled with access to technology in the schools, the ability to support more teachers, more frequently with immediate feedback on delivery of effective instructional strategies is now a reality. Technology opens the door for future pre-service teachers to receive unprecedented teacher training support and immediate feedback during their initial years of teaching. With the introduction of technology, pre-service teachers are now afforded the opportunity to have immediate support during critical classroom interactions between student and teachers. Coaches and pre-service teachers experience what is happening in the classroom in the most naturalistic state and in real-time. To ensure the use of in advance-structured, encouraging, immediate performance feedback, we proposed the use of a computer program for these specific feedback messages. Technically, sending a keyword was accomplished by pressing the right button on a computer interface that as a reaction triggered a computerized voice to transmit the keyword (Coninx, Kreijns & Jochems, 2013).The use of a computer program has additional benefits for practice. Computer-provided feedback (given by a coach) would seem to have several important advantages. Unlike feedback from a coach, this feedback can remain unbiased, nonjudgmental and irrespective of pre-service teacher characteristics. In addition, the interactive ability of computer-provided feedback has the potential for enhancing the quality and type of feedback that can be implemented. Thus, computer-provided feedback can, at least theoretically, be adapted to the learning styles and needs of each individual student, a goal that seldom is attained in a traditional classroom. Ideally, computer-provided feedback can be designed to be adaptive to the needs of the pre-service teacher, where a variety of levels of elaborative information can be incorporated into a computer-based program that could be accessed according to the achievement level of the student as well as the nature of the task for that particular pre-service teacher.
6.4.2
Instrumentarium
The implementation of synchronous coaching in schools appeared a bit more complicated, not only as a result of technical problems. The implementation of synchronous coaching was still underway in many of the schools that participated. The majority of schools have implemented only some of the elements of synchronous coaching. Therefore, in further studies, greater attention could be paid to (monitoring) the implementation process of the various elements of synchronous coaching and the possible influences of these factors on the results of the studies.
98
Conclusion and discussion Another future research domain can be longitudinal research. Follow-up data collection during a longer period of time after implementation could provide insights into the occurrence of possibly different results when synchronous coaching is implemented more solidly. In such longitudinal research, online measurements can be applied to obtain behavioral changes during pre-service teachers’ teaching performance. A larger sample of pre-service teachers and coaches would improve the representativeness of the research results. The adoption of a longitudinal research design to investigate the effects of might also be relevant.
6.5 Implications This research sought for empirical evidence that synchronous coaching is an effective teacher training tool for pre-service teachers. A major misunderstanding about synchronous coaching is that coaches can whisper immediate performance feedback on the spot, without preparations. However, synchronous coaching demands a thorough approach. This research aimed at a systematic approach of pre-service teachers. Coach and pre-service teacher have to discuss what will be observed and what kind of feedback will be provided in order to avoid cognitive overload and ambiguity. Coaches have to think about ways to provoke and support goal setting for pre-service teachers’ performance improvement. Maybe, synchronous coaching even demands more effort and attention of coaches than the regular approach in order to specify the feedback to the needs of the pre-service teachers. The coach’s expertise in supporting pre-service teachers and providing coaching is crucial (Evertson & Smithey, 2000). Coaches argued that not just anyone could be a coach. Additionally, the delivery of feedback to students appeared to be seamless and took preservice teachers no longer than 3 seconds to translate suggestions into practice. Pre-service teachers reported feeling at ease with having a coach communicating with BIE device and felt a sense of additional support with BIE. Pre-service teachers reported feeling supported and having an extra pair of eyes to watch the classroom. Finally, the time required for training pre-service teachers and coaches in the use of equipment and software was minimal. A trainings program originally planned for a one hour, only required approximately twenty minutes. So, synchronous coaching is increasingly implemented in teacher education, because it is believed to better prepare pre-service teachers. Pre-service teachers who require increased support during teaching can receive immediate feedback while implementing effective instructional strategies. The ability to pinpoint very specific strategies that can be effective in diverse classrooms with diverse students as well as their ability to support pre-service teachers as they implement those strategies becomes crucial in the development of pre-service teachers. Therefore, the questions of how to best implement synchronous coaching in educational pre-service teacher preparation programs and the effectiveness of synchronous coaching are important for educational researchers and policy makers. (Korner & Brown, 1952; Scheeler, 2008; Scheeler, et al., 2008; Rock et al., 2009b; Rock et al., 2009b) To conclude, synchronous coaching following a six steps approach and the use of keywords gains perceived satisfaction and behavioral improvement in pre-service teachers during their 99
Chapter 6 teacher training program. Teacher professional development activities like this should be implemented in schools more systematically, so that these activities lead to more professional teachers that are capable in providing quality education to their students.
References Bidwell, Charles E., (1965). The School as a Formal Organization, (pp. 972-1022) in Handbook of Organizations, edited by James G, March, Chicago: Rand McNally. Boe, E.E., Cook, L. H., & Sunderland, R. J. (2008). Teacher turnover: Examining exit attrition, teaching area transfer, and school migration. Exceptional Children, 75(1), 731. Bowles, E.P., & Nelson, R.O. (1976). Training teachers as mediators: Efficacy of a workshop versus the bug-in-ear technique. Journal of School Psychology, 14(1), 15-25. Brekelmans, M., Wubbels, Th., & Brok, P. den (2002). Teacher experience and theteacherstudent relationship in the classroom environment. In S. C. Goh & M. S. Khine (Eds.), Studies in educational learning environments: an international perspective (pp.73-99). Singapore: World Scientific. Coninx, N. S., Kreijns, C. J., & Jochems, W. M. G. (2013). The use of keywords for delivering immediate performance feedback on teacher competence development. European Journal of Teacher Education, 36(2), 164–182. Engelen, A.J.A. (2002) Coaching binnenstebuiten: een onderzoek naar coaching van docenten door docenten [Coaching form the inside out: a survey of coaching of teachers by teachers]. Unpublished doctoral dissertation: Nijmegen, The Netherlands: University Press. Evertson, C. M., &Smithey, M. W. (2001). Mentoring effects on proteges' classroom practice: An experimental field study. Journal of Educational Research, 93(5), 294-304. Farrell, A.C., & Chandler, D. (2008). Cooperating teachers’ impressions of the Whisper in my ear (WIME) and traditional communication feedback methods for physical education pre service teachers. Education and human development, 2(1), 1-9. Fry, J.M., & Hin, M.K.T. (2006). Peer coaching with interactive wireless technology between student teachers: satisfaction with role and communication. Interactive learning environments, 14 (3), 193-204. Giebelhaus, C. R., & Cruz, J. (1994). The mechanical third ear device: An alternative to traditional student teaching supervision strategies. Journal of Teacher Education, 45, 365-373 Giebelhaus, C.R. (1994). The bug-in-the-ear Device: An anlternative Student Teaching Supervision Strategy, paper presented at the association of teacher educators’ annual conference, Atlanta, GA. Goodman, J.I., & Duffy, M.L. (2007). Using BUGS to increase student participation. Teaching exceptional children plus, 3(4). Retrieved from http://scholarship.bc.edu /education /tecplus/vol3/iss4/art3. Goodman, J.I., Brady, M.P., Duffy, M.L., Scott, J., & Pollard, N.E. (2008). The effects of ‘bug-in-ear’ supervision on special education teachers’ delivery of learn units. Focus on autism and other developmental disabilities. 23(4), 207-216. Hashweh, M.Z. (2003). Teacher accommodative change. Teaching and Teacher Education, 19, 421-434. Hennissen, P., Crasborn, F., Brouwer, N., Korthagen, F., & Bergen, T. (2008) Mapping Mentor teachers’ roles in mentoring dialogues. Educational Research Review, 3, 168186.
100
Conclusion and discussion Horner, R. H., Carr, E. G., Halle, J., McGee, G., Odom, S., & Wolery, M. (2005). The use of single subject research to identify evidence-based practice in special education. Exceptional Children, 71, 165-180. Hunt, D.D. (1980). Bug-in-the-ear technique for teaching interview skills. Journal of medical education, 55, 964-966. Ingersoll, R. (2001) Teacher turnover, Teacher shortages, and the organization of schools. Philadelphia, Pennsylvania: University of Pennsylvania. Ingersoll, R. & Smith, T.M. (2004) Do teacher induction and Mentoring Matter? NAASP Bulletin, 88(638), 28-40. Ingersoll, R. and Strong, M. (2011). The Impact of Induction and Mentoring Programs for Beginning Teachers: A Critical Review of the Research. Review of Education Research, 81(2), 201-233. Kahan, D. (2002). The effects of a bug-in-the-ear device on intra lesson communication between a student teacher and a cooperating teacher. Journal of Teaching in Physical Education, 22(1), 86-104. Kersaint, G., Lewis, J., Potter, R., & Meisels, G. (2007). Why teachers leave: Factors that influence retention and resignation. Teaching and Teacher Education, 23(6), 775-794. Kirkpatrick, D. & Kirkpatrick, P. (2006). Evaluating Training Programs (3rd ed.). San Francisco, CA: Berrett-Koehler Publishers, Inc Korner, I., & Brown, W. (1952). The mechanical third ear. Journal of Consulting Psychology, 16(1), 81-84. Leary, T. (1957). Interpersonal diagnosis of personality: a functional theory and methodology for personality evaluation. New York, Ronald Press Co. Marzano, R.J., Pickering, D.J., & Pollock, J.E. (2001). Class room instruction that works: research based strategies for increasing student achievement. Alexandria, VA: Association for Supervision and Curriculum Development. Marzano, R.J., Pickering, D.J., & Pollock, J.E. (2009). Wat werkt in de klas. [What Works in the class room]. Vlissingen, The Netherlands: Bazalt. Price, A. T., Martella, R. C., & Marchand-Martella, N. (2002). A comparison of immediate feedback delivered via an FM headset versus delayed feedback on the inappropriate verbalizations of a student with ADHD. Education and Treatment of Children, 25, 159-171. Rathel, J.M., Drasgow, E. & Christle, C.C. (2008). Effects of supervisor performance feedback on increasing pre-service teachers’ positive communication behavior with students with emotional and behavioral disorders. Journal of Emotional and behavioral disorders, 16(2), 67-77. Rock, M.L., Gregg, M., Thead, B.K., Acker, S.E., Gable, R.A., & Zigmond, N.P. (2009). Can you hear me now? Evaluation of an online wireless technology to provide real-time feedback to special education teachers-in-training. Teacher Education and Special Education, 32(1), 64-82. Rock, M.L., Gregg, M., Howard, P.W., Ploessl, D.M., Maughn, S., Gable, R.A., & Zigmond, N.P. (2009). See me, hear me, coach me. Virtual bug-in-ear technology brings immediacy to professional development. National staff development council, 30(3), 24-32. Silverman, S. & Kimmel, J. (1972). The influence of immediate feedback on the behavior of teacher-in-training (Final Report No. ED068463). Gainesville, FL : University of Florida, and the National Center for Educational Research and Development. Scheeler, M.C., Lee, D.L. (2002). Using technology to deliver immediate corrective feedback to preservice teachers. Journal of behavioral education, 11(4). 231-241.
101
Chapter 6 Scheeler, M.C., McAfee, J.K., Ruhl, K.L., & Lee, D.L. (2006). Effects of corrective feedback delivered via wireless technology on preservice teacher performance and student behavior. Teacher education and special education, 29(1), 12-25. Scheeler, M.C., Macluckie, M., & Albright, K. (2008). Effects of immediate feedback delivered by peer tutors on the oral presentation skills of adolescents with learning disabilities. Remedial and special education, 11, 1- 10. Scheeler, M. C. (2008). Generalizing effective teaching skills: the missing link in teacher preparation. Journal of Behavioral Education, 17, 145-159. Scheeler, M. C., Bruno, K. & Grubb, E. (2009). Generalizing Teaching Techniques from university to K-12 classrooms: teaching pre-service to use what they learn. Journal of Behavioral Education, 18, 189-210. Smith, T. M., & Ingersoll, R. M. (2004). What are the effects of induction and mentoring on beginning teacher turnover? American Educational Research Journal, 41(3), 681-714. van der Mars, H. (1988). The effects of audio-cueing on selected teaching behaviors of an experienced elementary physical education specialist. Journal of Teaching in Physical Education, 8(1), 64-72. Veenman, S. (1984). Perceived problems of beginning teachers. Review of Educational Research, 54 (2), 143-178. Wade, Y.W. (2010). Increasing novice teacher support in 21st century classrooms: Induction and mentoring for beginning teachers through bug-in-ear technology. Unpublished doctoral thesis. [Orlando, Florida: University of Central Florida Walsh, C.M., Ling, S.C., Wang, C.S., & Carnahan, H. (2009). Concurrent feedback versus terminal feedback: it may be better to wait. Academic Medicine, 84(10), 54-57. Wubbels, T., Brekelmans, M., & Hooymayers, H. (1991). Interpersonal teacher behaviour in the classroom. In B.J. Fraser & H.J. Walberg (Eds.). Educational environments: Evaluation, antecedents and consequences (pp. 141-160). Oxford, England: Pergamon Press.
102
List of publications and presentations Peer-reviewed articles Coninx, N. S., Kreijns, C. J., & Jochems, W. M. G. (2013). The use of keywords for delivering immediate performance feedback on teacher competence development. European Journal of Teacher Education, 36(2), 164–182. Lord, D., & Coninx, N.S. (2012). E-mentoring in Vocational teacher education. Book chapter In Isaias, P., Ifenthaler, D., Sampson, D.G., Spector, J.M. (Eds.). Towards learning and instruction in web 3.0. Advances in cognitive and educational psychology. Springer, New York. Coninx, N., Kreijns, K., & Jochems, W. (submitted). Review of technology-enabled immediate performance feedback. A six steps approach to synchronous coaching for pre-service teachers. Coninx, N., Kreijns, K., & Jochems, W. (submitted). Perceived satisfaction of pre-service teachers due to synchronous coaching using Bug In Ear device. Coninx, N., Kreijns, K., & Jochems, W. (submitted). Observed behavioral change of preservice teachers due to synchronous coaching using Bug In Ear device. Peer-reviewed presentations Coninx, N.S, Kreijns, C.J., & Jochems, W.M.G. (2009). Optimizing wireless communication using cognitive load theory: The use of a wireless earpiece in coaching trainee teachers facilitating collaborative learning groups. Presentation at the Cognitive Load Conference, Heerlen, The Netherlands. Coninx, N.S, Kreijns, C.J., & Jochems, W.M.G. (2010). Perceived succes of immediate coaching of student teachers with WIME (Whisper In My Ear) device. Paper presented at SITE conference, San Diego, United States of America. Coninx, N.S, Kreijns, C.J., & Jochems, W.M.G. (2010). Changes in interpersonal behavior of student teachers due to immediate coaching with WIME (Whisper in My Ear) device. Poster presented at ICIRE conference, Boulder, United States of America. Coninx, N.S, Kreijns, C.J., & Jochems, W.M.G. (2010). Het success van synchroon coachen op de interpersoonlijke SBL competentie voor docenten in opleiding. Poster presented at ORD conference, Enschede, The Netherlands. Coninx, N.S, Kreijns, C.J., Jochems, & W.M.G. (2010). Het succes van onmiddellijke feedback voor beginnende leerkrachten door draadloze communicatie. Poster presented at ORD conference, Enschede, The Netherlands. Coninx, N.S, Kreijns, C.J., & Jochems, W.M.G. (2010). E-mentoring in vocational teacher education. Paper presented at CELDA conference, Timisoara, Romenia. Coninx, N.S., (2012). Kompas voor Excellentie: leraren in opleiding synchroon coachen. VELON VELOV conferentie, Antwerpen, België.
Professional articles and book (chapters) Coninx, N.S, Dirckx, P.A.L.M., Kreijns, C.J., & Jochems, W.M.G. (2009). Whisper in my ear, Vector, 11, 8-9.
103
List of publications and presentations Paroles, Coninx, N.S, & Dirckx, P.A.L.M. (2009). Congres Leraarschap in een ander perspectief. Meer, beter en moderner gereedschap in de tas. Vector, 11, 7. Coninx, N.S., Laureys, B., & Van Elsen, J. (2011). Cultuurgek 3. [Crazy by Culture 3] (3rd ed). Antwerpen, Belgium: Uitgeverij de Boeck. (ISBN: 9789045535579). Coninx, N.S., Laureys, B., & Van Elsen, J. (2011). Cultuurgek 3 handleiding voor de leerkracht. [Crazy by Culture 3, manual for teachers] (2nd ed). Antwerpen, Belgium: Uitgeverij de Boeck. (ISBN: 9789045535562). Coninx, N., Kreijns, K. & Jochems, W. (2011). Een fluisterend oor, het nut van synchroon coachen. [A whisper ear, the utility of synchronous coaching] (Onderzoekreeks #35). Zoetermeer, The Netherlands: Kennisnet. (ISBN: 978-90-77647-52-3). Lejeune G., & Coninx, N.S. (2011). Synchroon coachen, gestructureerd of ad hoc [Synchronous coaching, structured or ad hoc]. Internal rapport. Eindhoven, The Netherlands: University of Technology. Lejeune G., Coninx, N.S. (2011). Docenten coachen via koptelefoon. [Coaching teachers with ear piece]. Cursor (7), p.8. Coninx, N.S, & Dirckx P.A.L.M. (2012). Vijf generaties synchroon coachen. [Five generations of synchronous coaching]. Profiel, 1, 14-16. Professional presentations Coninx, N.S, Kreijns, C.J., & Jochems, W.M.G. (2008). Technology enabled coaching. Coaching of student teachers. Presentation at the ICO Summer School, Utrecht, The Netherlands. Coninx, N.S, Kreijns, C.J., & Jochems, W.M.G. (2008). Technology enabled coaching. Interne presentatie voor de Kenniskring, Sittard, Nederland. Coninx, N.S, (2009). De implicaties van Coachen met een oortje. Presentatie Trevianum, Sittard, The Netherlands. Coninx, N.S, (2009). De implicaties van Coachen met een oortje. Presentatie Connect College, Echt, The Netherlands. Coninx, N.S, Dirckx, P.A.L.M., Kreijns, C.J., & Jochems, W.M.G. (2009). Coaching met het ‘oortje’ van studenten. Workshop voor de Academische School, Echt, Nederland. Coninx, N.S, Kreijns, C.J., & Jochems, W.M.G. (2009). Coaching met het ‘oortje’ van (beginnende) docenten. Interne presentatie voor de Kenniskring, Sittard, Nederland. Coninx, N.S, Kreijns, C.J., & Jochems, W.M.G. (2009). Synchrone coaching van studenten met een ‘oortje’. Poster presented at the Research days of Fontys, Eindhoven, The Netherlands. Coninx, N.S, Martens, R., Kreijns, C.J., & Jochems, W.M.G. (2009). Onmiddellijke coaching van studenten. Workshop voor de Spring School van SURFacademy, Heerlen, Nederland. Coninx, N.S, Kreijns, C.J., & Jochems, W.M.G. (2009). Immediate coaching of student teachers with WIME (Whisper in my ear) device. Paper presented at The Summer School, Berlin, Germany. Coninx, N.S, Kreijns, C.J., & Jochems, W.M.G. (2009). Synchrone coaching van studenten met een ‘oortje’. Presentation at the Conference of The Academic School, Venlo, The Netherlands. Coninx, N.S, Kreijns, C.J., & Jochems, W.M.G. (2009). Tussentijds rapport. Veranderingen op interpersoonlijk vlak door coachen met het oortje. Interne presentatie voor de Kenniskring, Sittard, Nederland. Coninx, N.S, (2009). De implicaties van Coachen met een oortje. Presentatie Blariacum, Venlo, The Netherlands.
104
Coninx, N.S, Kreijns, C.J., & Jochems, W.M.G. (2009). Tussentijds rapport. Eerste resultaten van coachen met het oortje. Interne presentatie voor Promovendi-overleg, Eindhoven, Nederland. Coninx, N.S, Kreijns, C.J., & Jochems, W.M.G. (2009). Perceived success of immediate coaching of student teachers with WIME (Whisper in my ear) device. Paper presented at ICO Toogdag, Utrecht, The Netherlands. Coninx, N.S, Dirckx, P.A.L.M, Kreijns, C.J., & Jochems, W.M.G. (2009). Onmiddellijke coaching van studenten. Workshop voor de Onderwijsdagen van SURF, Utrecht, Nederland. Coninx, N.S, (2009). De implicaties van Coachen met een oortje. Presentatie Connect College, Echt, The Netherlands. Coninx, N.S, Kreijns, C.J., & Jochems, W.M.G. (2009). Tussentijds rapport. Eerste resultaten van coachen met het oortje. Interne presentatie voor FLOS Studiedag, Sittard, Nederland. Coninx, N.S, Kreijns, C.J., & Jochems, W.M.G. (2009). Immediate coaching of student teachers with WIME (Whisper in my ear) device. Paper presented at The Winter School, Rovaniemi, Finland. Coninx, N.S, Kreijns, C.J., & Jochems, W.M.G. (2009). De noodzaak van een docentonderzoeker. TOP bijeenkomst Fontys, Vught, Nederland. Coninx, N.S, Kreijns, C.J., Jochems, W.M.G. (2010). Rapport Amerika, opbrengsten voor Coachen met het ‘oortje’. Interne presentatie voor de Kenniskring, Sittard, Nederland. Coninx, N.S, Kreijns, C.J., & Jochems, W.M.G. (2010). Coachen met het ‘oortje’. Lerarenopleiding, Gent, België. Coninx, N.S, Kreijns, C.J., & Jochems, W.M.G. (2010). Overzicht van synchroon coachen. Presentatie tijdens FLOS studiedag, Sittard, The Netherlands. Coninx, N.S, Kreijns, C.J., & Jochems, W.M.G. (2010). Sychroon coachen werkt! Lunchlezing, Sittard, The Netherlands. Coninx, N.S (2010). De stem van de promovendi. FGS bijeenkomst, Eindhoven, The Netherlands. Coninx, N.S (2010). Synchroon coachen en iPads. PABO Eindhoven, Eindhoven The Netherlands. Coninx, N.S (2010). Synchroon coachen werkt! Lerarenopleiding, Ghent, Belgium. Coninx, N.S, Kreijns, C.J., & Jochems, W.M.G. (2010). Comments and questions on the relationship between students’ small group activities, time spent on self-study. Discussant presented at ICO Toogdag, Amstelveen, The Netherlands. Coninx, N.S (2010). De keywords die gebruikt worden in CONPAS. PABO, Eindhoven, Nederland. Coninx, N.S, & Dirckx, P.A.L.M. (2011). Synchroon coachen workshop. PABO, Eindhoven, Nederland. Coninx, N.S, & Dirckx, P.A.L.M. (2011). I. iFontys slotpresentatie, Eindhoven, Nederland. Coninx, N.S, & Dirckx, P.A.L.M. (2011). Synchroon coachen werkt. Sophianum, Gulpen, Nederland. Coninx, N.S, (2011). De methode van synchroon coachen. VOR ICT, Utrecht, The Netherlands. Coninx, N.S, & Dirckx, P.A.L.M. (2011). Synchroon coachen Basiscursus. PABO, Den Bosch, Nederland. Coninx, N.S. (2011). Synchroon coachen i.s.m. Training Individuele begeleidingsvaardigheden voor begeleiders van (aanstaande) leraren. Da Capo, Sittard, Nederland.
105
List of publications and presentations Coninx, N.S., & Dirckx P.A.L.M. (2011). Onmiddellijke feedback. Triveanum, Sittard, Nederland. Coninx, N.S., & Dirckx P.A.L.M. (2011). Onmiddellijke feedback. Graaf Huyn, Sittard, Nederland. Dirckx P.A.L.M., & Coninx, N.S. (2011). Synchroon coachen met. videocommunicatie. SURF Onderwijsdagen 2011, Utrecht, Nederland. Coninx, N.S., (2012). Het coachen van leraren via het elektronisch oortje. Onderzoek symposium, Sittard, Nederland. Dirckx, P.A.L.M; & Coninx, N.S., (2012). Synchroon coachen van docenten in opleiding met een oortje. HBO raad, Bussum, Nederland. Coninx, N.S., (2012). Synchronous coaching of student teacher with WIME (Whisper in my ear) device. Conference The path to becoming a vocational teacher, Tampere, Finland. Coninx, N.S. (2012). Samenwerking met enthousiaste kenniskringleden. Mini symposium, Sittard, Nederland.
106
Measuring effectiveness of synchronous coaching using Bug-In-Ear device of pre-service teachers Summary Teachers are identified as the most important factor to influence the quality of education in schools. The quality of the professional development of these teachers largely depends on the support that is given during all three stages of their career; teacher training (training of preservice teachers before entering the classroom as a fully responsible teacher), induction (a process of coaching and support during the first teaching practice in an actual classroom of a particular school) and continuing professional development (an in-service process for practicing teachers). The teaching profession has alarmingly high rates of dropouts during the induction stage where they become an independent teacher by going through the first years of confrontation with reality. A lot of newly qualified teachers leave the job early. There are several causes for these fall-out rates. Teaching has become more demanding. The increased diversity of student population presents new challenges for teachers. Children come to school from various cultural and socioeconomic backgrounds, each bringing different kind of experiences to school requiring additional specific competences of the teachers. Beginning teachers have consistently reported feeling unprepared and the lack of support during their induction phase. These teachers reported struggling with students’ diverse backgrounds and fail to develop effective instructional strategies so to support students from these backgrounds. Schools and teacher preparation programs are acting as professional development schools and are responsible for the guidance of pre-service teachers doing their teaching practice. The school as a learning environment may reduce the gap between practice and theory. There is now more emphasis placed on the school as a learning environment where teachers’ professional learning can be supported. The high fall-out rates costs schools and teacher preparation programs time and money as they continually must find and prepare new teacher candidates. Also, high rates of turnover of induction programs contribute significantly to the attrition of beginning teachers in education. Supporting and optimizing teacher professional development more systematic and empirically validated research is needed. The scope of this study is limited by the different aspects discussed. The support of pre-service teachers, or the lack thereof, is first felt during the internship at the professional development schools. Due to this reality-chock experience, the focus of this research has fallen on the teacher training. Because of the turn-over issues, the research has turned on to the effectiveness and efficiency of this teacher training. Problems of pre-service teachers, their causes and solutions can be addressed depending on the perspective taken. The perspective used in this research is the micro perspective in which the individual teacher and his work for the class are taken as viewpoint. Improvement of teacher training is within this perspective the considered central solution. The teacher training stage is considered a part of the teacher’s development, whether or not it takes place within the first years of employment. The aim of this PhD research is to measure the effectiveness of synchronous coaching using bug-in-ear technology (BIE technology). The general research question is: "What are the effects of synchronous coaching using BIE by pre-service teachers?" In order to address the general research question, four more specific research questions were posed: 107
Summary
1) 2) 3) 4)
How do we organize synchronous coaching? Does this systematic approach reduce cognitive load and ambiguity? What are the effects at the level of reaction of Kirkpatrick? What are the effects at the level of behavioral change of Kirkpatrick?
In answering the first research question, the theoretical review study of published material results in the following definition of synchronous coaching: In education, synchronous coaching is a method for giving immediate performance feedback with certain characteristics (specific, corrective, encouraging) by using BIE on targeted performance. These feedback messages are structured and discussed in advance. Most commonly a cooperating teacher sits in the back of an actual class room en sends the feedback to a pre-service teacher. A six steps approach with a training program for sending immediate performance feedback is preferable. Rather than imposing a new construct definition on the field, the first study addresses the need for construct clarification using existing descriptions of common characteristics to create a snapshot of the synchronous coaching literature. Through a qualitative, literature-based approach, a definition of synchronous coaching is developed. A total of 29 descriptions has been extracted from 79 sources. These have been systematically analyzed for the characteristics that help explain construct meaning. A six steps approach has been developed to summarize the current understanding of synchronous coaching: 1) Taking care of two prerequisites, a) Both coach and pre-service teacher must be willing to participate b) Getting parental permission if pupils in the class room are being filmed or interviewed for research purposes 2) Offering a training program for using the BIE device 3) Discussing performance of the pre-service teacher 4) Intervention: the actual usage of the BIE device in the class room: preferable four times with each lesson ending in a reflection. In this reflection the next targeted behavioral improvement and feedback codes should be discussed 5) Fading the usage of BIE (keeping the BIE device in the ear but not giving immediate performance feedback by coach and at the end not using the BIE device at all) 6) Evaluation of performance of pre-service teacher. Analysis of this model provides better understanding of the current state of synchronous coaching, identifies gaps in the literature, and informs and directs future research on synchronous coaching. The next step in this research is that a (pilot) in vivo study has been carried out in a real life setting of a classroom with pupils, using this six step approach to show its effectivity. Although the definition of synchronous coaching is a useful model for teacher educators, especially with the trainings program, the innovative element of the BIE device, namely prestructured keywords, needs to be more elaborated to determine the effectiveness of synchronous coaching. The increased amount of cognitive processing required performing a task while receiving, interpreting and responding to immediate feedback may impose high extraneous cognitive load on pre-service teachers. Does this systematic approach reduce cognitive load and ambiguity (the multi-interpretability of the keywords)?
108
The large majority of the key elements incorporated in the theoretical study addresses issues or areas that have received significant research attention over a long life time span. By presenting them in a clarifying scheme including a training program with five sub steps more insight in relevant contributing factors has been provided.
1) Acquaint the pre-service teacher with wearing the BIE device. This is done in a lesson where no messages should be whispered 2) Then, determine the teaching behaviors to be developed and select appropriate keywords 3) In the same lesson a week later, give immediate specific performance feedback (selected keywords) by using the BIE device 4) After the lesson, conduct a semi structured interview per couple (coach and pre-service teacher) 5) Fill in the questionnaire in and send it back to the researcher.
The results of this in vivo study supported the hypotheses that the use of structured keywords for delivering immediate performance feedback is more beneficial than the ad hoc mode of delivering performance feedback, in terms of cognitive load and ambiguity for pre-service teachers. In the ad hoc mode, the coach is free in which feedback messages are whispered to the pre-service teacher. Usually this ends up that the feedback messages consist out of more than one sentence. In the structured mode, coach and pre-service teacher always have to select and discuss the keywords in advance that they are going to use, therefore diminishing the chance of ambiguity. By structuring the messages and only using the selected keywords it is expected that cognitive load (i.e. extraneous load) can be reduced. Now that synchronous coaching and its implementation method are defined, an additional study is conducted to measure the effects of synchronous coaching on the first level of effectiveness of Kirkpatrick, namely perceived satisfaction. To this end, a literature search has been conducted on perceived satisfaction of synchronous coaching, combined with an in vivo study with 80 respondents (pre-service teachers and their coaches). The literature findings showed that according to pre-service teachers synchronous coaching is more supportive, is providing more useful feedback, and is more encouraging than asynchronous ways of coaching where the feedback is given afterwards. The synchronous coaching group showed a significantly greater satisfaction for three functions, supportiveness, giving feedback and encouragement, than the asynchronous coaching group. The other two functions, discussing suggestions and discussing the effects on pupils, were also positive but not statistically more positive. The results of the in vivo study indicated that synchronous coaching is a satisfying technique for coaching pre-service teachers. Overall, pre-service teachers in the current study rated synchronous coaching favorably with respect to three functions of coaching namely supportiveness, giving useful feedback en encouragement. This is consistent with the findings of the extensive literature search. The unique niche filled by synchronous coaching with the use of BIE-technology during preservice teachers’ field experiences makes a valuable tool for coaches to provide immediate performance feedback when it comes to the satisfaction of the pre-service teachers. The last part of the research was to measure the effects of synchronous coaching on the behavioral change of pre-service teachers (third level of evaluation of a trainings program by Kirkpatrick). The content of the behavioral changes was composed of three categories: class
109
Summary room management, instructional activation of pupils and pedagogical relationship between teacher and pupils. The results confirmed early findings regarding the benefits of synchronous coaching to alter effective teaching behavioral changes. Given that teacher’s effectiveness is one of the most important factors affecting pupil’s performance, it is essential that researchers collect outcomes that document those changes in teacher behavior. Pre-service teachers who received synchronous coaching, became more cooperative and had more ascendancy in their class room. It is apparent that asynchronous coaching do not guarantee behavior changes in the class room, which is the accepted goal of pre-service teachers training. It is expected that their pupils will show higher affective and cognitive outcomes. What teachers do in the class room is noted by pupils and that it really does matter. This study underscored the fact that synchronous coaching is a practical tool with which to strengthen the teaching and learning process of pre-service teachers. Synchronous coaching is believed to better prepare pre-service teachers. Novice teachers who require increased support during the initial years of teaching can receive immediate feedback while implementing effective instructional strategies. The ability to pinpoint very specific strategies that can be effective in classrooms with diverse students as well as their ability to support novice teachers as they implement those strategies becomes crucial in the development of beginning teachers’ dispositions and delivery of instruction. Therefore the questions of how to best implement synchronous coaching in teacher preparation programs and the effectiveness of synchronous coaching are important for educational researchers and policy makers. With the use of BIE technology, pre-service were able to increase desired teaching behaviors. The findings of this PhD research underline the important role of immediate performance feedback with innovative technology in increasing pre-service teachers’ perceived satisfaction and effective behavioral improvement. To conclude, synchronous coaching following a six steps approach and the use of keywords gains perceived satisfaction and behavioral improvement in pre-service teachers during their induction program. Teacher professional development activities like this should be implemented in schools more systematically, so that these activities lead to more professional teachers that are capable in providing quality education to their students.
110
Meten van effectiviteit van synchroon coaching door BugIn-Ear technologie van leraren-in-opleiding Samenvatting Leerkrachten worden als de belangrijkste invloedsfactor op de kwaliteit van het onderwijs gezien. De kwaliteit van de professionele ontwikkeling van deze docenten is in belangrijke mate afhankelijk van de steun die ze krijgen gedurende de drie fases in hun loopbaan, namelijk hun initiële opleiding tot leraar (ze zijn dan een leraar-in-opleiding), de introductie in het leraarsambt (ze zijn dan beginnende leraren) en de verdere professionalisering (ze zijn dan ervaren leraren). Het beroep van leraar kent een alarmerend hoog aantal uitvallers tijdens de inductiefase, de tweede fase in hun loopbaan waarin ze na een initiële opleiding kennis maken met de realiteit van leerkracht zijn. De reden is vaak dat zij zich onvoldoende gesteund voelen tijdens het opdoen van praktijkervaring. De kwaliteit van leraren is een vereiste voor hoge kwaliteit van onderwijs dat dan op zijn beurt medebepalend is voor Europees lange-termijn competiviteit en mogelijkheid voor groei. Het is dus belangrijk dat tijdens alle fases de ontwikkeling van de leraren ten volle ondersteund wordt. Beginnende leraren hebben een aantal redenen voor het verlaten van het beroep na hun eerste jaar. Lesgeven is veeleisender geworden. De toegenomen diversiteit van de leerlingenpopulatie geeft nieuwe uitdagingen voor docenten. Leerlingen komen uit verschillende culturele en sociaal-economische achtergronden, die elk hun specifieke eisen stellen aan de competenties van de leerkracht. Beginnende leraren hebben aangegeven onvoldoende voorbereid te zijn en voelen een gebrek aan steun. Ze worstelen met de diverse achtergronden en het lukt ze niet om effectieve educatieve strategieën te ontwikkelen die het ondersteunen en onderwijzen van deze leerlingen te vergemakkelijken. Er wordt aangenomen dat de zogenoemde reality-shock die ervaren wordt tijdens de start als leraar, een gevolg is van de beperkte praktische relevantie van de lerarenopleidingen. Scholen en lerarenopleidingen worden verantwoordelijk gehouden voor de begeleiding van leraren-inopleiding en voor het opdoen van ervaring tijdens praktijklessen. De school als leeromgeving kan de kloof tussen praktijk en theorie verminderen en daarmee de reality-shock wegnemen. Hoge uitvalcijfers kosten tijd en geld omdat scholen en lerarenopleidingen voortdurend nieuwe leerkrachten moeten vinden en opleiden. De ondersteuning van leraren-in-opleiding, of het ontbreken daarvan, wordt voor het eerst duidelijk tijdens de stage gedurende de lerarenopleiding en is de focus van dit onderzoek. Het ondersteunen en optimaliseren van de professionele ontwikkeling van leraren-in-opleiding heeft meer systematisch en empirisch gevalideerd onderzoek nodig. Dit onderzoek richt zich vooral op de effectiviteit en efficiëntie van een stage, met name de begeleiding ervan. Het doel van dit promotieonderzoek is om de effectiviteit van synchrone coaching als hulpmiddel tijdens een stage te optimaliseren, waarbij gebruik gemaakt wordt van Bug-InEar-technologie (BIE-technologie), kortweg een oortje. De algemene onderzoeksvraag is: "Wat zijn de effecten van synchrone coaching met het gebruik van BIE-technologie op leraren-in-opleiding?". Om de algemene onderzoeksvraag te beantwoorden, werden vier meer gedetailleerde onderzoeksvragen opgesteld: 1) Hoe moeten we synchroon coachen uitvoeren? 111
Samenvatting 2) Vermindert deze systematische aanpak de cognitieve belasting en ambiguïteit van de leraren-in-opleiding? 3) Wat zijn de effecten in termen van het reactieniveau van Kirkpatrick? 4) Wat zijn de effecten op het niveau van gedragsverandering van Kirkpatrick? Bij het beantwoorden van de eerste onderzoeksvraag resulteerde de theoretische literatuurstudie in de volgende definitie van synchrone coaching: Synchrone coaching in onderwijsopleidingen is een methode voor het geven van onmiddellijke feedback over de prestaties van leraren(-in-opleiding) door het gebruik van BIE. De feedback heeft daarbij de volgende kenmerken: specifiek, correctief en bemoedigend en is gericht op specifieke gedragingen van de leraar. De feedbackberichten worden gestructureerd en vooraf besproken. Meestal zit een samenwerkende leraar achter in de klas en stuurt deze de feedback naar een beginnende leerkracht. Een zes-stappen aanpak inclusief een opleidingsprogramma voor het verzenden van onmiddellijke feedback heeft de voorkeur. Op basis van een analyse van wetenschappelijk literatuur is de zes-stappen benadering van synchrone coaching ontwikkeld, samengevat in volgende stappen: 1) Het invullen van twee voorwaarden, a) Zowel de coach als de leraar-in-opleiding moeten bereid zijn om deel te nemen b) De toestemming van de ouders moet worden verkregen als leerlingen in de klas worden gefilmd of geïnterviewd voor onderzoeksdoeleinden 2) Het geven van een trainingsprogramma voor het gebruik van een BIE apparaat 3) Het bespreken van de prestaties van de leraar-in-opleiding 4) Interventie: het daadwerkelijke gebruik van het BIE apparaat in de klas: bij voorkeur vier keer waarbij elke les eindigt in een reflectie. In deze reflectie moeten de verbeteringen in gedrag en de bijhorende feedback-codes worden besproken 5) Het geleidelijk verminderen van het gebruik van BIE (het apparaat in het oor houden, terwijl er geen onmiddellijke feedback over de prestaties door de coach wordt gegeven tot aan het moment dat er niet meer met behulp van het BIE apparaat moet worden gecoacht) 6) Evaluatie van de prestaties van de leraar-in-opleiding. Analyse aan de hand van dit model geeft inzicht in de huidige stand van zaken over synchrone coaching en identificeert hiaten in de literatuur. Uitgaande van de definitie van synchrone coaching is een bruikbare methodiek voor lerarenopleidingen ontwikkeld met behulp van het vernieuwende element van BIE technologie, waarbij pre-gestructureerde trefwoorden (i.e., de feedback-codes), worden gehanteerd om de effectiviteit van synchrone coaching te optimaliseren. Synchroon coachen vereist dat de leraar-in-opleiding tijdens het uitvoeren van een taak als leraar voor de klas onmiddelijke feedback in de vorm van codes ontvangt, interpreteert en op basis daarvan zijn doceergedrag aanpast. Doordat de gewone feedback berichten (die doorgaans uit vele zinnen kan bestaan en uitgesproken worden door de coach) vervangen zijn door feedback-codes wordt een te hoge cognitieve belasting van de leraar-in-opleiding vermeden. Doordat in het trainingsprogramma de feedback-codes worden uitgelegd (zie hierna) wordt vermeden dat tijdens het gebruik de leraar-in-opleiding de feedback-codes op meerdere manieren zou kunnen uitleggen (vermijding van ambiguïteit). De gewone manier van het geven van feedback berichten wordt in dit onderzoek aangeduid als de ad hoc modus en de manier waarbij gebruik wordt gemaakt van feedback-codes de gestructureerde modus.
112
Op basis van een literatuurstudie is een trainingsprogramma ontwikkeld bestaande uit vijf sub stappen : 1) Laat de leraar-in-opleiding eerst kennis maken met het dragen van het BIE apparaat. Dit gebeurt tijdens een les waarin geen feedback-codes moeten worden gefluisterd 2) Vervolgens worden de te ontwikkelen competenties bepaald en de juiste feedback-codes geselecteerd. Bespreek de vooraf bepaalde lijst van feedback-codes om problemen en bijbehorende oplossingen te identificeren 3) Geef in dezelfde les een week later onmiddellijke specifieke prestatiefeedback (geselecteerde feedback-codes) 4) Voer, na de les, een semi gestructureerd interview per paar uit (coach en leraar-inopleiding) 5) Vul de vragenlijst in en stuur het terug naar de onderzoeker. De resultaten geven aan dat het gebruik van feedback-codes voor het leveren van onmiddellijke prestatiefeedback beter is dan de ad hoc-modus als het gaat over het geven van minder cognitieve belasting en ambiguïteit voor beginnende docenten. Voor ambiguïteit lijkt dit aannemelijk omdat in de gestructureerde modus coach en beginnende leraar de feedback-codes moeten selecteren en bespreken voordat ze worden gebruikt. Dit vermindert de kans op ambiguïteit. Ook voor de verlaging van de cognitieve belasting lijkt dit aannemelijk omdat de feedback-codes belangrijk korter zijn dan de vele zinnen van de coach wat de interpretatie van de feedback berichten eenvoudiger maakt. Nadat synchroon coachen is gedefinieerd, een stappenplan is vastgesteld en de feedbackcodes zijn bepaald, werd een aanvullende studie uitgevoerd naar de effecten van synchrone coaching op het eerste niveau van effectiviteit van Kirkpatrick, namelijk waargenomen tevredenheid. Hiertoe werd een literatuuronderzoek uitgevoerd, gecombineerd met een in vivo onderzoek met 80 respondenten (leraren-in-opleiding en hun coaches). De resultaten uit de in vivo studie ondersteunde de literatuurbevindingen waarin volgens de leraren-in-opleiding synchrone coaching meer ondersteunend is, meer nuttige feedback verstrekt en bemoedigender is dan de traditionele manier van coaching waarbij de feedback wordt pas achteraf wordt gegeven. De synchrone coaching groep toonde een significant grotere tevredenheid voor drie onderdelen van lesgeven (ondersteuning geven, het geven van feedback en aanmoedigen) dan de traditionele coaching-groep. De andere twee onderdelen, het bespreken van suggesties en bespreken van de effecten op de leerlingen, waren ook positief, maar statistisch gezien niet significant. In het algemeen beoordeelden leraren-in-opleiding in de huidige studie synchroon coachen gunstig met betrekking tot drie onderdelen van coaching, namelijk het ondersteunen, het geven van nuttige feedback en het aanmoedigen. Deze aspecten zijn gerelateerd aan de redenen die genoemd worden waarom beginende leraren het beroep leerkracht te verlaten. Dit is consistent met de bevindingen van het uitgebreide literatuuronderzoek. Het laatste deel van dit onderzoek omvatte het meten van de effecten van synchrone coaching op gedragsveranderingen van leraren-in-opleiding (derde niveau van de evaluatie van een trainingsprogramma). De gedragsverandering is van toepassing op drie categorieën: klassenmanagement, educatieve activering van de leerlingen en de pedagogische relatie tussen leraar en leerlingen. De resultaten bevestigen eerdere bevindingen met betrekking tot de voordelen van synchrone coaching versus de traditionele manier van coaching om effectief didactische gedragsveranderingen te bewerkstelligen. Gezien het feit dat de effectiviteit van
113
Samenvatting de leraar een van de belangrijkste invloedsfactoren op de prestaties van leerlingen is, is het essentieel dat onderzoekers de resultaten van deze gedragsveranderingen documenteren. Leraren-in-opleiding die synchrone coaching ontvingen, waren coöperatiever en hadden meer overwicht in hun klas. Deze studie onderstreepte het feit dat synchrone coaching een praktisch instrument is om het didactisch leerproces van leraren-in-opleiding te versterken omdat het de leraren-in-opleiding ondersteunt tijdens dit proces. Synchrone coaching maakt het mogelijk leraren-in-opleiding beter voor te bereiden op het echte leven als leraar. Met het gebruik van BIE technologie kunnen beginnende leraren hun gedrag verbeteren. Meer in het bijzonder, leraren-in-opleiding kunnen door feedback hun gedrag sneller aan specifieke studenten in leesonderwijs aanpassen. Het vermogen om specifieke strategieën in de klas met verschillende studenten effectief te laten zijn en het vermogen om beginnende docenten te ondersteunen bij het implementeren van deze strategieën is cruciaal in de ontwikkeling van beginnende leerkrachten. Synchrone coaching met een zes-stappen aanpak en het gebruik van feedback-codes geeft waargenomen winsten in tevredenheid en gedragsverbetering van leraren-in-opleiding tijdens hun stageprogramma. Dergelijke professionele ontwikkelingsactiviteiten zouden meer systematisch moeten worden ingevoerd op scholen, zodat het leidt tot meer professionele docenten die kwalitatief goed onderwijs aan hun leerlingen leveren.
114
Curriculum Vitae Nele Coninx was born on the 30 June 1979 in Hasselt, Belgium. She obtained her secondary school diploma in Latin-mathematics. In 1997 she started at the Free University Brussels (Belgium) studying the master Social and Cultural Agogy. In 2001 she obtained the master of science with her master thesis ‘Exploration of the coaching modalities of in-service teachers’, together with a first level teaching qualification. From 2001 to 2007 Nele was a secondary teacher and she taught different courses in the domain of social and cultural sciences. Later, she became team leader and coach of preservice teachers. She did also innovative projects for students and colleagues teachers. In 2006 she graduated on Coaching of in-service teachers, a bachelor after master, at the EHSAL (Management School) Brussels, Belgium. Facilitated by Fontys University of Applied Sciences, Nele started at the end of 2008 her PhD project at the Eindhoven School of Education. She was guided by her promotors from the Eindhoven School of Education (ESoE) and the Scientific Centre for Teacher Research (Open University)). During her PhD project, she was also employed in various projects in collaboration with schools. These projects all focused on the innovative teacher at the workplace. She presented her research at national (Velon, ORD) as well as international (CLC, Site, Celda, ICIRE) conferences. Currently she is working at the Teacher Education Department of the Catholic University of Applied Science Limburg (KHLim), Belgium, as a coordinator of research and development. Nele’s interests are innovation projects in education and especially the role of the teacher therein.
115
116
Eindhoven School of Education Dissertation series Sande, R.A.W. van de (2007). Competentiegerichtheid en scheikunde leren: over metacognitieve opvattingen, leerresultaten en leeractiviteiten. Hooreman, R.W. (2008). Synchronous Coaching of Trainee Teachers, an experimental approach; ICO Dissertation Series No. 188 (Doctoral thesis, Eindhoven School of Education, Eindhoven University of technology, Eindhoven, Nederland) Rajuan, M. (2008). Student teachers’ perceptions of learning to teach as a basis for supervision of the mentoring relationship. Raessens, B.A.M. (2009). De E-kubus: een analysemodel voor curricula. Rohaan, E.J. (2009). Testing teacher knowledge for technology teaching in primary schools. Oemar Said, E. (2009). De Da Vinci Case: een onderzoek naar de relaties tussen vernieuwende leeromgevingen en de motivatie en regulatievoorkeuren van leerlingen in het MBO. Koopman, M. (2010). Students’ goal orientations, information processing strategies and knowledge development in competence-based pre-vocational secondary education. Mittendorff, K.M. (2010). Career conversations in senior secondary vocational education. Crasborn, F.J.AJ. en Hennissen, P.P.M. (2010). The skilled mentor. Mentor teachers’ use and acquisition of supervisory skills. Bragt, C.A.C. van (2010). Students’ educational careers in Higher Education: A search into key factors regarding study outcome. Bakker, G. de. (2010). Allocated online reciprocal peer support via instant messaging as a candidate for decreasing the tutoring load of teachers. Vos, M.A.J. (2010). Interaction between teachers and teaching materials: on the implementation of context-based chemistry education. Bruin-Muurling, G. (2010). The development of proficiency in the fraction domain. Kraemer, J.M. (2011). Oplossingsmethoden voor aftrekken tot 100. Cornelissen, L.J.F. (2011). Knowledge processes in school-university research networks. Stiphout, I.M. van (2011). The development of algebraic proficiency. Saeli, M. (2012). Teaching programming for secondary school. Putter-Smits, L.G.A. de (2012). Science teachers designing context-based curriculum materials: developing context-based teaching competence. Ketelaar, E. (2012). Teachers and innovation: On the role of ownership, sense making and agency. Dehing, F. (2012). Preparing students for workplace learning in higher engineering education. Vrijsen-de Corte, M.C.W. (2012). Researching the teacher-researcher. Practice-based research in Dutch professional development schools. Doppenberg, J.J. (2012). Collaborative teacher learning: settings, foci and powerful moments. Linden, W. van der (2012). A design-based approach to introducing student teachers in conducting and using research. Diggelen, M.R. van (2013). Effects of self-assessment procedure on VET teachers’ compenties in coaching students’ reflection skills. Gorissen, P.J.B. (2013). Facilitating the use of recorded leactures: Analysing students’ interaction to understand their navigational needs. Keuvelaar – van den Bergh, L. (2013). Teacher feedback during active learning: The development and evaluation of a professional development programme. Pillen, M.T. (2013). Professional identity tentions of beginning teachers. Marée, T.J. (2013). Scripted collaborative enriched skeleton concept mapping to foster meaningful learning.
117
Eindhoven School of Education Dissertation series Goméz Puente, S.M. (2014). Design-based learning: exploring an educational approach for engineering education.
118
