Professional Development

A Guide to

Teaching Strategies GOLD® Interrater Reliability Certification

The Purposes of Teaching Strategies Interrater Reliability Certification Teaching Strategies GOLD ® is an authentic, ongoing, observation-based assessment system that relies heavily on your judgment as a teacher. The information you collect every day by observing children in the context of meaningful experiences makes this instrument robust and effective. After analyzing the evidence you gathered to document each child’s knowledge, skills, and behaviors, you evaluate each child’s progress. You identify each child’s levels of development and learning at given points in time; track progress over time; and compare the child’s knowledge, skills, and behaviors with widely held expectations for children of the same age or class/grade. Your evaluations inform the decisions you make when planning learning experiences for individual children and for your group as a whole. It is important to use Teaching Strategies GOLD ® reliably, both to scaffold children’s learning and because your program administrator will probably use your data in some way for reporting purposes. One criticism of any observation-based assessment system is that evaluators’ judgments are somewhat subjective. Administrators of programs using this type of assessment tool have voiced concerns about whether evaluators (teachers) can make accurate, consistent decisions about children’s levels of development and learning. They wonder whether teachers who observe the same situation would rate the children’s skills and behaviors the same way and, more importantly, whether they would make the same rating decisions as the developers of the tool would make. When you use assessment results to inform instructional decisions, accurate ratings enable you to choose effective teaching strategies. When you know you are using an assessment tool reliably, you can be confident about your classroom decisions. You know that you are interpreting evidence of children’s development and learning in ways that enable you to plan for and respond appropriately to all of the children in your classroom. To address administrators’ concerns and support your efforts, Teaching Strategies has taken steps to check and enhance the accuracy of your assessment ratings through a process that leads to interrater reliability certification. In addition to affording practice in using the tool, the process measures the degree of agreement between your ratings and those of Teaching Strategies master raters. We developed Teaching Strategies interrater reliability certification to • increase the accuracy and consistency of your judgments as you use Teaching Strategies GOLD® • improve planning for individual children by helping you assess children accurately • identify professional development needs by using data collected during the certification process Teaching Strategies interrater reliability certification is neither designed nor intended to evaluate you as a teacher. Its purpose is to enhance one of your many competencies that benefit the children you work hard to support each day in your classroom.

2

A Guide to Teaching Strategies GOLD® Interrater Reliability Certification

What Interrater Reliability Certification Means After completing Teaching Strategies GOLD ® implementation training, you may choose to strengthen your implementation skills and receive interrater reliability certification from Teaching Strategies. Interrater reliability is the extent to which the ratings assigned by two or more individuals (observers/ raters) agree. It addresses the consistency with which a rating system is used by multiple persons. Being awarded Teaching Strategies interrater reliability certification means that your ratings of children’s knowledge, skills, and behaviors agree (within an acceptable range) with the ratings determined by Teaching Strategies GOLD ® developers. The online certification process gives you multiple opportunities to rate children’s knowledge, skills, and behaviors. You will opt to evaluate portfolios for children in one of seven age-groups: birth through age 2; age 3 through kindergarten; mixed ages; birth through age 2, including children with disabilities; preschool, including children with disabilities; preschool, including dual-language learners; or preschool, including dual-language learners and children with disabilities. Each portfolio was reviewed by Teaching Strategies master raters, who agreed on a rating for the child’s knowledge, skills, and behaviors in relation to each objective and dimension. Your ratings will be compared with theirs. Agreement with a checkpoint rating means that your rating is within a range of ±1 of the master rating. For example, if you enter a rating of 5 for a dimension and the master rating is 6, the ratings are within the acceptable range of agreement. If you enter 4 but the master rating is 6, the ratings do not fall within the acceptable range of agreement. To be awarded interrater reliability certification, at least 80 percent of your ratings in each of six areas of development and learning must agree with the master ratings (within the acceptable range of ±1).

3

A Guide to Teaching Strategies GOLD® Interrater Reliability Certification

The Interrater Reliability Certification Process You will begin the interrater reliability certification process by evaluating the portfolios of three children. Those initial evaluations are called “Round 1.” The portfolios include enough evidence to rate each child’s knowledge, skills, and behaviors in six areas of development and learning: Social– Emotional, Physical, Cognitive, Language, Literacy, and Mathematics. The children whose portfolios you will evaluate have been attending their programs for a while and have not had excessive absences, so you should not choose ‘Not Observed’ for any rating.” If at least 80 percent of your Round 1 ratings agree with the master ratings in each area of development and learning, you will have earned certification. If not, you will be able to evaluate three more portfolios, concentrating on the areas in which you did not reach 80 percent agreement. If there are areas in which you do not reach 80 percent agreement during Round 2, you will be able to evaluate a third set of portfolios, focusing on those areas. You will be able to continue with additional rounds until you have reached agreement of at least 80 percent in each area. The number of rounds you may undertake is not limited.

Certification Testing Agreement The Teaching Strategies GOLD ® interrater reliability certification test is to be taken only by the individual named in the Teaching Strategies account. As an examinee, you should neither solicit nor accept any assistance during the testing process. In order to take the test, you will be required to certify that 1) you are the individual who is registered to take the test and 2) you are completing the test without assistance from any source.

Preparing for Certification Teaching Strategies recommends that you complete Teaching Strategies GOLD ® implementation training before beginning the certification process. Teaching Strategies provides two forms of implementation training: an online course (available to all Teaching Strategies GOLD ® users via the professional development tab in the Teaching Strategies dashboard) or a 2-day, in-person professional development session. Training sessions introduce participants to the structure of Teaching Strategies GOLD ®, the objectives for development and learning, the assessment cycle, and evaluation of child portfolios. Additional preparation is recommended and available through the “Practice Checkpoints” that are part of the online professional development module. In “Practice Checkpoints,” users can review portfolios, enter checkpoint levels, and then compare their ratings to those established by master raters. Users will also see a justification statement that explains the reasoning behind the master rating. Finally, users can prepare for the certification test by reading Objectives for Development & Learning to become more familiar with the 38 objectives, the dimensions, and the progressions of development and learning.

Testing Timeline You may take as much time as you need to complete each round. There are no time limits. Your starting and completion dates for each round will be listed under the professional development tab, which you access from in your Teaching Strategies dashboard. We suggest that you take time between rounds to review the objectives for development and learning in the areas in which you did not reach 80 percent agreement. Also review the PDFs that are provided at the end of each round that provide additional guidance and suggestions for improving your accuracy. This also is a good time to review the Teaching Strategies GOLD ® implementation training modules.

4

A Guide to Teaching Strategies GOLD® Interrater Reliability Certification

How to Complete the Test Getting Started To access the interrater reliability certification screens, log in to your TeachingStrategies.com account. Once you have logged in, go to your dashboard. Then click the professional development tab at the top of the screen and select Interrater Reliability from the dropdown menu. You may undertake interrater reliability certification in English or Spanish. To complete the test in English, continue to the bottom of the English Overview screen. To complete it in Spanish, click Complete the Interrater Reliability Certification test in Spanish /Tomar el Examen de Certificación de Consistencia entre Evaluadores en español.

5

A Guide to Teaching Strategies GOLD® Interrater Reliability Certification

Overview Screen At the bottom of the Overview screen, you will be asked to read and agree to a verification statement. If you do not understand the statement for any reason, please consult with your administrator or send an e-mail to [email protected] for clarification.

Choosing an Age-Group Choose one of the following age-groups as the focus for certification testing: birth through age 2; age 3 through kindergarten; mixed ages; birth through age 2, including children with disabilities; preschool, including children with disabilities; preschool, including dual-language learners; or preschool, including dual-language learners and children with disabilities.

6

A Guide to Teaching Strategies GOLD® Interrater Reliability Certification

Starting a Round This screen shown below includes a list of evaluation rounds to be completed, your progress to date, and details about each round. Starting and completion dates are listed as well as the areas you need to strengthen in order to be certified. Click Round 1 to begin.

7

A Guide to Teaching Strategies GOLD® Interrater Reliability Certification

Instructions The Instructions screen provides step-by-step directions for navigating the other interrater reliability certification screens. When you have finished reading the instructions, click CONTINUE at the bottom of the Instructions screen.

8

A Guide to Teaching Strategies GOLD® Interrater Reliability Certification

Checkpoint by Area The Checkpoint by Area for Interrater Reliability Certification screen is similar to screens in Teaching Strategies GOLD ®. The names of three children are listed on the left side of the chart, and six areas of development and learning are listed across the top. The boxes in the middle of the chart show your completion status in each area. Refer to the key on the right side of the screen for details about the colors of the boxes.

To begin or continue rating children’s knowledge, skills, and behaviors, click on any box in the chart. This will take you to the “Checkpoint” screen for a child. You may review and change your ratings at any time before you submit them for scoring. Once you have completed checkpoint ratings for all children in an area and have reviewed your ratings, click Submit to see your results.

9

A Guide to Teaching Strategies GOLD® Interrater Reliability Certification

Checkpoint The Checkpoint screen for each child presents a progression that shows an objective, a dimension, indicators, examples, and ranges of widely held expectations. Click Show under Examples on the left side of the Checkpoint screen to see the examples for the progression. Review the documentation and choose a level. Proceed to the next dimension and continue until you have completed all the dimensions for that child in that area. You may stop at any time by clicking Save and Close in the lower right corner of your screen. You may leave the test, log out of your Teaching Strategies account, and return to the certification screens at any time.

10

A Guide to Teaching Strategies GOLD® Interrater Reliability Certification

Viewing Results The Interrater Reliability Certification: Your Results screen will show your agreement score for each round and area. You will be given an overall score for each area of development and learning. If you need to complete another round of evaluations for areas in which you did not achieve at least 80 percent agreement with the master ratings, you will be given suggestions for preparing for another round. Please note that the results screen shows a total agreement score for each round and area. To maintain the integrity of the certification, the results screen does not provide specific information about how each of your item ratings compared with each master rating. Click on the Round “x” Results link at the top of each column in the chart to review results for previous or current rounds. At the end of each round, you can review PDFs with additional guidance on improving your accuracy and examples specific to the areas you did not pass.

11

A Guide to Teaching Strategies GOLD® Interrater Reliability Certification

Viewing Your Incorrect Answers For each area you did not pass, you will be able to review the answers you submitted that were not within the acceptable range of reliability. Click on the gray “X” in the checkpoint by area chart.

You can begin to review each progression. Any answers that were not within the acceptable range of reliability will show as a gray X in the progression. To maintain the validity and integrity of the test, we do NOT show the answers for progression in which you did meet the acceptable range of reliability.

12

A Guide to Teaching Strategies GOLD® Interrater Reliability Certification

Certificate of Completion Once you have achieved a passing score in all six areas of development and learning, you will receive access to a link that enables you to print a certificate of completion for your records. Your certification will be valid for 3 years from the date you passed the certification test. The results of your certification test will also be available to your administrator through the Teaching Strategies GOLD ® system.

13

A Guide to Teaching Strategies GOLD® Interrater Reliability Certification

We hope that you find this guide to be useful. If you have further questions about how to complete this course or require additional support, just e-mail us at [email protected] or call our support line at 1.866.736.5913. Thank you for using Teaching Strategies GOLD® in your classroom!