observer teacher planning and preparation link to syllabus and spec excellent planning individual learning needs identified and addressed

Faculty One Initials

etc Initials

T1 T2 T3 T4 T1 T2 T3 etc 1

n

% planning and preparation link to syllabus and spec excellent planning individual learning needs identified and addressed

linking of teaching strategies to learning outcomes appropriate resources to support learning embedding cross cutting themes bilingual strategies outside visits and oppportunities outside class wide ranging resources used including ILT learning needs and differentiation on plan managing learning links to / review previous learning effective questioning techniques active learning - variety of t&l strategies pairs and group work managed well, promoting autonomy timed activities use of moodle and/or ILT effective use of resources/ quality of resources peer learning

linking of teaching strategies to learning outcomes appropriate resources to support learning embedding cross cutting themes bilingual strategies outside visits and oppportunities outside class wide ranging resources used including ILT learning needs and differentiation on plan managing learning links to / review previous learning effective questioning techniques active learning - variety of t&l strategies

learning environment - learners in control of learning logcial structures and sequences effective classroom management encouraging participation voice quality Detailed Feedback and praise

learning environment - learners in control of learning logcial structures and sequences effective classroom management encouraging participation voice quality Detailed Feedback and praise

subject knowledge/ up to date vocational expertise differentiation in evidence

subject knowledge/ up to date vocational expertise differentiation in evidence

pairs and group work managed well, promoting autonomy timed activities use of moodle and/or ILT effective use of resources/ quality of resources peer learning

challenge and pace/ challenging thinking equal opportunities clarity of instructions effective communication student motivation and interest. Inclusive literacy/ numeracy strategies employed links to vocation/ vocational skill study skills tips clear aims and objectives shared with learners buzz and tangible learner enjoyment effective ending to session - review learning support for learners ice breakers high expectations extension activities all learning styles learner needs catered for motivating learners passion for subject skills development evident creativity and imagination assessment and evaluation peer assessment student self assessment formative assessment techniques used summative assessment (homeworks/ assignments) learning outcomes reviewed and met evidence of effective learning assessment criteria made clear to learners/ modelling answers link to key skills exemplars used from other students appropriate assessment good knowledge of student progress - ILPs/ Action plans used high quality student portfolios - work and notes

1

challenge and pace/ challenging thinking equal opportunities clarity of instructions effective communication student motivation and interest. Inclusive literacy/ numeracy strategies employed links to vocation/ vocational skill study skills tips clear aims and objectives shared with learners buzz and tangible learner enjoyment effective ending to session - review learning support for learners ice breakers high expectations extension activities all learning styles learner needs catered for motivating learners passion for subject skills development evident creativity and imagination assessment and evaluation peer assessment student self assessment formative assessment techniques used summative assessment (homeworks/ assignments) learning outcomes reviewed and met evidence of effective learning assessment criteria made clear to learners/ modelling answers link to key skills exemplars used from other students appropriate assessment good knowledge of student progress - ILPs/ Action plans used high quality student portfolios - work and notes

constructive and detailed written feedback on student work good knowledge of students use of tracking sheet for learner progress clear skills development relationship with clients technology utilised for assessment link to world skills peer review professionalism punctuality and attendance timekeeping - start and finish of lesson effective student teacher relationship/ mutual respect/ rapport appropriate use of support staff tutor's enthusiasm/ commitment smartly dressed confident professionalism learner praise for teacher learning environment environment stimulating learning suitability of space free of distractions health and safety issues quality relevant wall displays good practice examples on walls students assessing risk bilingual wall displays total number of features grade given number of strengths identified number of areas for development identified

constructive and detailed written feedback on student work good knowledge of students use of tracking sheet for learner progress clear skills development relationship with clients technology utilised for assessment link to world skills peer review professionalism punctuality and attendance timekeeping - start and finish of lesson effective student teacher relationship/ mutual respect/ rapport appropriate use of support staff tutor's enthusiasm/ commitment smartly dressed confident professionalism

2 1 1

0

0

0

0

0

0

learning environment environment stimulating learning suitability of space free of distractions health and safety issues quality relevant wall displays good practice examples on walls students assessing risk bilingual wall displays total number of features grade given number of strengths identified number of areas for development identified * similar lessons observed - similar format of comments made

Areas for development identified on lesson observation Differentiation extension tasks /homeworks Develop strategies to support individual learner needs identified challenging students and using questioning Revisit learning outcomes to check met - plenary activities use timed activities

1

Areas for development Differentiation extension tasks /homeworks Develop strategies to support individual learner needs identified challenging students and using questioning Revisit learning outcomes to check met - plenary activities use timed activities

Share aims of session with the learners / clarify tasks link session to previous learning Pace of session - delivery/ timings and content/ challenge Ensure learning outcomes in the session reflect the activity use up to date paperwork - session plans

Share aims of session with the learners / clarify tasks link session to previous learning

More student input/ active learning/ practical tasks engage all learners and get them to participate develop self assessment to check learning in session/ peer assessment opps closer portfolio monitoring of progress - ILPs developing learner feedback to be more purposeful / develop own feedback skills use of moodle/ ICT Linking to vocational practice Keep subject knowledge accurate and up to date develop how to embed literacy skills further

More student input/ active learning/ practical tasks engage all learners and get them to participate develop self assessment to check learning in session/ peer assessment opps closer portfolio monitoring of progress - ILPs? developing learner feedback to be more purposeful / develop own feedback skills use of moodle/ ICT Linking to vocational practice Keep subject knowledge not accurate and up to date

Pace of session - delivery/ timings and content/ challenge Ensure learning outcomes in the session reflect the activity use up to date paperwork - session plans

Observation Process Review The College’s 2009-2010 observation cycle

The research.. •

78 observations – performance review and teaching excellence observations

•

15 observers (from every faculty, included Assistant Directors)

•

Every positive point recorded on observation form

•

Final strengths and weaknesses recorded (training needs analysis and a record of what observers perceive as important teaching and learning strengths)

Finding 1 Uncertainty of grading at Grade 1 and 2 - leading to over grading

Grade 1 Grade 2 Grades 3&4

Estyn 2009

College Observations

26% 56% 18%

38% 47% 15%

Finding 1 Uncertainty of grading at Grade 1 and 2 – leading to over grading Trend towards over grading reflected in number of strengths/ weaknesses

Average number of Strengths identified

Average number of Areas for Development identified

Grade 1

20

2

Grade 2

18

2

Grade 3&4

11

4

15% of all sessions had no identified areas for development. (3 sessions at grade 2 and 12 sessions at grade 1).

Finding 2 Observers are over rewarding traditional teaching techniques and are under rewarding new important learning techniques • • • •

Most popular strengths recorded: ‘Links to syllabus and excellent planning’ (grade 1 67%, grade 2 70%, grade 3&4 64%) ‘Excellent subject knowledge’ (grade1 73%, grade 2 78%) ‘Effective Feedback and praise’ (grade1 57%, grade 2 62%, grades 3&4 45%)

‘effective feedback and praise’. • • •

Merged with rapport with students/ enthusiasm for teaching. More grade 2 sessions and a high number of grade 3&4 session had this as a learning feature. Very little recorded about how the feedback developed in terms of the individual’s learning (how and why it was effective).

Assessment for Learning: 77% of grade 1 sessions ‘formative assessment techniques’ present But how is it evidenced? In Grade 1 sessions: • 10% peer assessment, • 23% self assessment • 23% peer learning, • 23% learners had good knowledge of own progress and used ILPs • 20% used model answers and explained assessment criteria

Little exploration of deeper formative assessment learning features that should be associated with grade 1 sessions. Lack of pedagogical understanding from the observers is the root cause of the over grading issue.

Finding 2 Observers are over rewarding traditional teaching techniques and are under rewarding new important learning techniques

•

Observers commented less frequently on more up to date key learning features.

•

Most focus for observers was on subject knowledge and compliance of paperwork. (40% of all observations identified session planning as area for development.)

•

This indicates a need for more focus on ‘the quality of learning’ in the classroom, in line with CIF.

•

It also indicates a training need for observers to ensure to be up-to-date with the most recent teaching and learning techniques.

Finding 3 The current observation form needs updating to include the Learning Core and to consider grading judgements The observation form is out of date and needs updating to include areas such as: • ESDGC • Bilingualism learning environment, Welsh context • Learners applying their own experiences to learning • Developing flexible learning • Blended learning and using technologies • Developing learners’ skills in literacy, numeracy and ICT in relation to the specialist area • Opportunities for learners to formulate and focus on their learning goals • Learners involved in assessment of their own learning

Consider final grading grid for improved grading judgements.

Finding 4 The observation form provides important data for monitoring the quality of observations, teaching and learning and training needs Need to strengthen compliance, named observation forms must be sent to HR by observers Benefits: • a triangulation for training needs • a useful quality measure to understand grading decisions. • Improved performance management process – observer training

Observer training Aim: • More secure grading decisions and improved observation process •

To recognise what is outstanding/ sector leading

Outputs: • An agreed taxonomy of learning techniques •

Observer training on new education developments

•

Embedding bilingualism and basic skills into learning

•

Further roll out across College of taxonomy of learning/ grade 1 features

•

Training with teams to develop/ recognise sector leading practice

How to Analyse Observation Forms – Quantifying Qualitative Judgements   (This document refers throughout to the spread sheet in Annex B ‘Lesson Observation Analysis Blank  Template’)  Preparatory work:  1. Ensure that all observation forms are collected centrally for the purpose of review.  2. Collate the observation forms by observer and label each observation form ‘T1, T2’ etc. On  the spread sheet the initials of the observer will be marked and the teachers observed will  be known by this reference (c.f. Lines 1‐3 of spread sheet).  3. On the spread sheet list the typical teaching and learning features identified in the College’s  observation forms (c.f. Column A in spread sheet)  Observation Analysis:  4. Read through each observation form.  5. Mark ‘1’ on the spread sheet for each identified feature in the observation form, for  example ‘effective links to the syllabus and the specification’ (c.f. example line 5 column B in  spread sheet).  6. Add any new features not collated in step 3 to the spread sheet as you go. These may be  important features omitted from your College observation form. This acts as a gap analysis.  7. If your College observation form has a summary at the end of the form for strengths  identified, these strengths can be highlighted (example column B line 40 in spread sheet).  These can then be counted (line 93, column B).  8. Areas for development can also be counted (line 94, column B). These also can be listed as  training needs (lines 97 on). A separate sheet can be created from this as we know from this  spread sheet that T1 needs training on questioning techniques (line 100, column B). We  could create a list of all the teachers who need training on questioning in a separate sheet.  Outputs of the Analysis:  •

•

• •

From this analysis it is possible to identify trends in how observers judge observations. Are  the observers rewarding deeper pedagogical features or purely managerial features such as  paperwork? Are they addressing the impact on learning or focussing on teaching?   It is also possible to correlate the number of features mentioned by the observer. If the  College observation form  has a summary of strengths, these can also be counted. Both can  be compared with grade decisions (line 92). Is there a clear distinction between grade  boundaries in terms of the number of features identified, and the number of strengths?   This analysis can also support the review of the observation form, how well does the  observation form allow the observers to make accurate judgements on learning?  This analysis is important in identifying the training needs for observers, both individually  and collectively. It allows the College to check the patterns of features identified by  observers and identify where observers may need more training, e.g. assessment for  learning techniques. It also allows the College to check individual observers and put in  support for high risk observers through tandem observations. 

•

 

The analysis provides training needs information for teaching and learning/ staff  development managers and it is possible to check how well these training needs are then  picked up at performance review, giving a useful triangulation. 

Analysis of Classroom Observations at Yale College 2009-2010

The grading decisions of our observation team and recommendations for further developing the observation process

by Carys Davies

Summary of findings:

Finding 1 Uncertainty of grading at Grade 1 and 2 – leading to over grading

Finding 2 Observers are over rewarding traditional teaching techniques and are under rewarding new important learning techniques

Finding 3 The current observation form needs updating to include the Learning Core and to consider grading judgements

Finding 4 The observation form provides important data for monitoring the quality of observations, teaching and learning and training needs

Overview of Study 78 observations – performance review and teaching excellence observations 15 observers (these were from every faculty and included Assistant Directors and managers with cross-college roles)

2

Finding 1: Uncertainty of grading at Grade 1 and 2 – leading to over grading College observers graded 12% more grade 1s than Estyn (2009). This indicates continuing uncertainty with grading at the boundaries between grade 1 and grade 2 and a trend towards over grading. This is reinforced by the lack of difference between the average number of strengths commented on in the observation forms at grade 1 and grade 2 (fig.2). As this paper will show, observers are not generally recognising the higher level learning features when they observe and are rewarding more traditional teaching techniques, which Estyn placed as grade 2 features. 15% of all sessions had no identified areas for development. (3 sessions at grade 2 and 12 sessions at grade 1). Fig. 1 Overview of grading

Grade 1 Grade 2 Grades 3&4

Estyn Inspection October 2009 26% 56% 18%

78 internal observations (2009-2010 cycle) 38% 47% 15%

Fig. 2 Average number of strengths identified in observation forms:

Grade 1 Grade 2 Grade 3&4

Average number of Strengths identified 20 18 11

Average number of Areas for Development identified 2 2 4

3

Finding 2: Observers are over rewarding traditional teaching techniques and under rewarding new important learning techniques

Fig. 3 shows that emphasis by observers across the College is on traditional teaching techniques with less exploration of new learning and assessment for learning techniques. Most popular strengths recorded were: ‘Links to syllabus and excellent planning’ (grade 1 67%, grade 2 70%, grade 3&4 64%) ‘Excellent subject knowledge’ (grade1 73%, grade 2 78%) ‘Effective Feedback and praise’ (grade1 57%, grade 2 62%, grades 3&4 45%) Lack of understanding on more up to date learning techniques can be seen through the observers rewarding of ‘effective feedback and praise’. This was commonly merged with generalised points about rapport with students and enthusiasm for teaching. Interestingly, more grade 2 sessions and a high number of grade 3&4 session had this as a learning feature. However, there was very little comment across the observations about how the feedback developed in terms of the individual’s learning or even how and why it was effective. On the whole, observers noted that grade 1 teachers generally had more: effective links to previous learning, effective questioning techniques, active learning strategies effective classroom management. good pace and there was a high degree of challenge in class Although these features can indicate excellent practice, they could well be present in grade 2 sessions. In general, there was little indication from observation records as to why session achieved a grade 1 as opposed to a grade 2. The small variance in the average number of strengths at grade 1 and 2 (fig. 2) mean that it was not true that grade 1 sessions necessarily had more strengths than grade 2 sessions.

4

Fig.3 Most popular areas identified as strengths across 78 observations Grade 1 %

Grade 2 %

Grades 3&4 %

link to syllabus and spec

67

70

64

excellent planning

63

65

27

Aims and objectives shared with learners

70

46

18

links to / review previous learning

80

49

64

effective questioning techniques

93

49

36

active learning - variety of t&l strategies

60

46

9

effective use of resources/ quality of resources

70

62

45

effective classroom management

73

24

45

Detailed Feedback and praise

57

62

45

subject knowledge/ up to date vocational expertise

73

78

45

challenge and pace/ challenging thinking

60

54

18

formative assessment techniques used

77

57

55

summative assessment (homeworks/ assignments)

67

46

45

punctuality and attendance

67

46

18

timekeeping - start and finish of lesson

90

73

55

effective student teacher relationship/respect/ rapport

100

73

64

suitability of space

57

59

0

health and safety issues

53

49

27

quality relevant wall displays

50

32

18

Planning and preparation

Managing Learning

Assessment and Evaluation

Professionalism

Learning Environment

5

Observers commented less frequently on more up to date key learning features. This indicates that there needs to be far more focus on the quality of learning in the classroom in line with the developments of the new Common Inspection Framework in Wales. It also indicates a significant training need for observers who may not be up-to-date with the most recent teaching and learning techniques.

It is interesting to note the planning and preparation feature in fig 4. Observers commented in 70% of grade 1sessions that ‘learning outcomes were shared with learners’ (fig 3.) and this seems to be an important grading feature for our observers. However the observers only rewarded 13% of grade 1 sessions with ‘linking teaching strategies to the learning outcomes’. The presumption here by our observers is that if learning outcomes are shared, then they are present and that is sufficient. However, surely at grade 1, teachers should be not only sharing learning outcomes with learners but driving those learning outcomes through the planned teaching strategies. Again, this provides an indication of the scale of training needs for the observation team.

The same analysis can also be supplied for assessment for learning techniques, where 77% of grade 1 sessions had a comment on ‘formative assessment techniques’ present in the session. Triangulate this with the data from grade 1 sessions (data from figs. 4 and 5): 10% evidenced peer assessment, 23% self assessment present 23% evidence of peer learning, 23% learners had good knowledge of own progress and used ILPs 20% used model answers and explained assessment criteria The conclusion of this is that the observers across the observations often made generalised points on formative assessment without exploring these deeper formative assessment learning features that would be associated with grade 1 sessions. I believe that this lack of pedagogical understanding from the observers is the root cause of the over grading issue.

6

Fig. 4 Other key learning features Grade 1 %

Grade 2 %

Grades 3&4 %

Individual learning needs addressed

17

3

0

Linking teaching strategies to learning outcomes

13

0

9

Appropriate resources to support learning

23

19

9

pairs & group work managed well, promoting autonomy

40

30

9

Peer learning

23

16

0

learning environment - learners in control of learning

33

30

18

Differentiation in evidence

47

35

18

peer assessment

10

8

0

student self assessment

23

14

0

learning outcomes reviewed and met

40

19

9

evidence of effective learning

43

38

0

33

43

18

Planning and Preparation

Managing Learning

Assessment and Evaluation

Learning Environment environment stimulating learning

7

Finding 3. The current observation form needs updating to include the Learning Core and to consider grading judgements Other areas observers commented on which were not currently included as prompts on the observation form: the learning core embedding and supporting literacy and numeracy in the classroom links to vocational skills and employment bilingualism learner centred teaching - assessment for learning techniques and support commenting on student work Using the LLUK standards in Wales as a guideline for observation form construction, there needs to be the following elements included for comment: ESDGC (AS2) Bilingualism learning environment and the Welsh context (AS3, BS1) Awareness of Equality and Diversity issues, including learner needs and safety (AS6) Learners applying their own experiences to learning (BS2) Developing flexible learning (BS2) Evidencing use of appropriate communication technologies with learners including different media and emerging technologies (BS3) Opportunities for learners to contextualise learning to the vocational area and the wider world (CS1) Apply own specialist area teaching strategies (CS3) Developing learner expertise in literacy, bilingualism, numeracy and ICT in relation to the specialist area (CS3) Opportunities for learners to formulate and focus on their learning goals (DS2) Learners involved in assessment of their own learning (ES1, ES3)

The observation form and observers also need to explore in more depth how ILT is planned and used to support learning and assessment in the class, i.e. one teacher was rewarded on the observation form with ‘effective use of IT’ for marking the electronic register in class. Also if grading remains an issue, it would seem sensible to look at an additional section in grid format for observers to reflect on their comments and to develop judgements leading to grading based on the evidence on the observation form.

8

Fig 5. Other Teaching and Learning Elements not included in the Observation Form Grade 1 %

Grade 2 %

Grades 3&4 %

embedding cross cutting themes

20

16

0

bilingual strategies

17

5

9

learning needs and differentiation on plan

27

14

9

student motivation and interest. Inclusive

47

46

9

links to vocation/ vocational skill

33

41

9

Support for learners

33

22

9

literacy/ numeracy strategies employed

10

5

0

assessment criteria made clear/ modelling answers

20

0

0

good knowledge of progress - ILPs/ Action plans used

23

8

0

use of tracking sheet for learner progress

13

3

0

high quality student portfolios - work and notes

10

8

0

Constructive detailed written feedback on student work

7

16

0

Good practice examples on walls

3

3

0

Bilingual wall displays

3

3

0

Planning and Preparation

Managing Learning

Assessment and Evaluation

Learning Environment

9

Finding 4. The observation form provides important data for monitoring the quality of observations, teaching and learning and training needs This study reveals the lack of compliance in the current system for observation form submission. HR receives a proportion of observations carried out across the College and then submit them to CPPD where they are stored in a secure area. If there is to be an improvement in the quality of observations, there needs to be stronger systems in place to ensure that all observations are sent to HR. As fig. 6 shows, the observation form is also an excellent source of evidence for training needs. And although these should be placed on the PD3, a desk top collection of training needs from the observation form provides excellent triangulation and also an indication of what observers are valuing as important teaching and learning features that need further development. This is important data as it monitors the quality ultimately of the performance review, showing us the teaching and learning priorities of the observers and the limits of their understanding in teaching and learning. An example of this is that only 2 members of staff had development of literacy and numeracy strategies to support learners as an area for development. Under the new Common Inspection Framework, using techniques in class to support these skills will be a fundamental feature of a grade 1 session. Also, the teaching excellence observation process should be updated in line with the new Common Inspection Framework Fig 6. Areas for training and further development identified by observers Number of staff /78 Staff below level 2 on observation (TLM support?)

12

Session planning (elements needing work)

31

Differentiation

14

Moodle and flexible learning

9

ESDGC

8

Professionalism

4

Peer and self assessment techniques

3

Active learning techniques

6

Linking learning to vocation and employment

4

Developing varied and effective resources

9

Developing feedback skills

10

Developing questioning techniques

10

Developing group and peer learning

8

Improving subject knowledge

1

Support for attaining grade 1 (TLM support?)

4

Developing Welsh context and a bilingual learning environment

4

Developing literacy and numeracy techniques to support learners

2

10

Conclusions and Recommendations Observer Training 1. Recognition of core observation team including ADs, cross college managers and TLMs. Outcome: Trained observers and the ability to create a standardised observation process 2. Significant training of the core observation team to recognise and understand up to date teaching and learning techniques. Outcome: a clear understanding of what constitutes an outstanding session. 3. ADs also need a thorough understanding of their own specialist area pedagogy and have up to date knowledge of how technology can be used innovatively to develop learning in their area. This includes how to develop a blended learning environment through ‘Gold standard’ use of Moodle. Outcome: sector leading practice (potential for awards and showcasing) 4. Ensure that all teachers, even those judged to be outstanding, have some area identified for further development and exploration on their observation form. Outcome: cultural shift to continuing professional development and enrichment for experienced staff 5. Continue to support the core observation team with tandem observations and observation support. Outcome: quality assured observation process with secure grading 6. Ensure that all of the core observation team are aware of how TLMs can be used to support areas for development and training needs arising from the observation. Outcome: improved resource usage of expertise

Wider Staff Training 7. Continued communication/training with all staff to further embed the learning core, cross cutting themes and the bilingual learning environment. Outcome: staff being more inspection ready. 8. Training of staff on techniques to enable them to support literacy and numeracy needs in class. Outcome: improved results for learners

Observation Form Development and Monitoring Processes 9. Redesign the College observation form to reflect new learning agendas. Consider developing grading criteria in a grid to encourage more accurate grading decisions. Outcome: improved grading decisions 10. Consider how the New Common Inspection Framework will impact on the teaching excellence process and the judgments made. Outcome: aligning the teaching excellence process with the new Common Inspection Framework 11. Continue to monitor and analyse observation forms annually as a triangulation for training needs and as a useful quality measure to understand grading decisions. Strengthen compliance so that a full copy of each observation form is sent to HR. Outcome: improved monitoring of observation decisions and training needs

11