2012 Lessons Learned from Scoring Student Work Range finding, scoring, and data review of questions from the Reading Measurements of Student Progress (MSP) and the High School Proficiency Exam (HSPE) provide the opportunity to see hundreds of student responses at each grade level and to evaluate data summarizing student performance. Using this information, the reading assessment team is able to share observations about student responses and areas of reading where students appear to be struggling. This document describes the type of questions students will encounter on the assessment and gives examples of student responses that earn full points. All Resources mentioned throughout this document can be found at: http://www.k12.wa.us/Reading/Assessment/default.aspx
Online and Paper/Pencil For the third year, schools could take the reading MSP online. About 50 percent of students in grades 6-8 (reading and mathematics), 25 percent of students in grades 4 (reading and math) and 5 (reading, mathematics, and science), and 15 percent of grade 3 (reading and math) participated. A survey OSPI conducted showed that 82 percent of students preferred online to paper/pencil.
Question Types The three types of questions on the MSP and HSPE are; completion, multiple-choice, and short answers. It is always important to read the question carefully. Often words that help focus the student’s answer will be in bold or italics. A question may also direct the student to where the answer will be found, for example a caption, certain paragraph number, or heading. Multiple-choice questions ask students to choose between either three (grades 3-5) or four (grades 6-HS) answer options, are worth one point, and make up approximately 70 percent of the assessment. A common multiplechoice error happens when students do not read carefully through each option before choosing the best option. Multiple-choice questions are written so that students must think about each option before deciding on the correct answer, there are no throw-away options. Students should return to the text to verify the answer is correct before making their final selection. Completion questions ask students to write an answer on either a blank line or inside a graphic organizer, are worth one point, and make up approximately 5 percent of the assessment. Completion questions are intended to be answered with a word or brief detail from the passage. A common completion question error happens when students leave the box blank, include information that is already given in another part of the graphic organizer, give vague answers, or refer to the wrong section of the passage. Short Answer questions ask students to write a short answer using support from the passage, are worth two points, and make up approximately 25 percent of the assessment. Short answers are scored using a rubric of possible text-based details or reasonable inferences and conclusions that support the question. Rubrics are developed by Washington State teachers during item writing and refined while reviewing student responses during the pilot range finding. Student responses earn full points by including at least two bullets from the rubric.
OSPI Reading Assessment 12/19/12
1
Students who earn full points on short answer questions often use the following strategies: Cite details by using specific wording from the passage in their answer. Supply more details than asked. For example, if asked to support with two details, a student will provide three or four. Organize their answers using bullet points or numbers. Answer all parts of the question. Students who do not earn full points on short answer questions: Do not read the stem carefully. (See example #1) Do not complete the question. When asking for two details, only one is given. (See examples #3 and #8) Use background knowledge instead of text-based details. Some students may have previous knowledge on a passage topic; however to keep it consistent for all students, points are only given for text-based details. (See example #4) Give a vague answer. Students will use words such as stuff/things etc. that are not specific. She was nice… is too vague. What did she do that was so nice? (See examples #5 and #7) Restate the question but do not give text-based details in their response. (See examples #2 and #6) Do not refer back to the specific section of a passage. (See example #4 and #9)
OSPI Reading Assessment 12/19/12
2
Example #1 Read the sentence below and provide a synonym for the word flock. The flock of birds flew over the grassy meadow and blocked the light from the setting sun. Possible answers are: assembly, crowd, A group of animals Synonym: _______________________________________ swarm, group, and This student provided herd. a definition instead of a synonym to replace the word. This is an example of a completion item.
Example #2
This student used information that is already mentioned in the Summarizing Statement box so does not earn a point.
This is another example of a completion item. The complete passage and rubric for this question can be found at: Grade 6, “Every ‘Buddy’ Loves to Read!” item #6.
OSPI Reading Assessment 12/19/12
3
Example #3 This student missed the opportunity to add a second detail.
This student’s answer is too vague. To earn a point, this answer needs a detail explaining how they (the egret) look for fish.
This example is a 0 response. The complete passage and rubric for this question can be found at: Grade 4, “Fishing is for the Birds,” item #10.
This question directs students to a specific section of the text.
Example #4
This example is a 0 response.
This student used background information rather than text details to answer the question.
The complete passage and rubric for this question can be found at: Grade 4, “The Story of (Human) Calculators,” item #5. OSPI Reading Assessment 12/19/12
4
Example #5
How long?
This student’s answer just misses by giving general statements that do not include text based details.
What was their goal?
This example is a 0 response. The complete passage and rubric for this question can be found at: Grade 6, “Testing the Wheel,” item #8
Example #6 What example from the text proves he was devoted to his work and his wife?
This example is a 0 response.
This student missed a point opportunity by repeating that he is devoted.
The complete passage and rubric for this question can be found at: Grade 6, “The Tailor’s Wish,” item #7.
OSPI Reading Assessment 12/19/12
5
Example #7 What did Jingles do to prove he was brave?
This student is using background information to answer the question.
This example is a 0 response. The complete passage and rubric for this question can be found at: Grade 7, “The Returning Cats,” item #2.
Example #8
This student’s answer is a great detail from the text but missed the second score point by only including one detail.
This example is a 1 response. The complete passage and rubric for this question can be found at: Grade 7, “The Friendship Orchard,” item #11
OSPI Reading Assessment 12/19/12
6
Example #9
Watch for clues like bold or italicized print.
This student’s answer is not from the specific section asked for in the question.
This example is a 0 response. The complete passage and rubric for this question can be found at: Grade 7, “Glowing Talk,” item #9.
OSPI Reading Assessment 12/19/12
7
Grades 3, 4 and 5 The reading assessments across grades 3, 4 and 5 are very similar. Students read 4-5 passages balanced between informational and literary genres and answer 30 questions covering the learning strands and targets listed below. The only difference between each grade-level test is that the rigor of the passages increases.
Strand
Comprehension
Analysis
Target 1 2 3 4 5 6 7 8 9 10 11
Description Main Idea Summary/Summarize Infer/ Predict Vocabulary Sequence Literary Elements / Text Features Compare / Contrast Cause / Effect Analyze Usefulness of Resources Author’s Purpose Extend Beyond the Text
3rd grade
4th grade
rd
th
Statewide, 3 grade students showed their best performance in: stand-alone questions analyzing Areas that are in need of attention are: sequencing (target 5) constructed response (short answer and completion) poetry
Statewide, 4 grade students showed their best performance in: poetry informational text short answer Areas that are in need of attention are: analyzing resources (target 9) biographies
5th grade Statewide, 5th grade students showed their best performance in: stand-alone questions short answer for literary Areas that are in need of attention are: informational text completion questions analyzing
Data for specific district and school trends within the text-type and strand can be accessed through tools on the State Report Card site under drop down menus. http://reportcard.ospi.k12.wa.us/summary.aspx?year=2010-11.
OSPI Reading Assessment 12/19/12
8
Grades 6, 7 and 8 The reading assessments across grades 6, 7 and 8 look very similar. Students read 5-6 passages balanced between informational and literary genres and answer 35 questions covering the learning strands and targets listed below. The level of rigor for passages continues to increase.
Strand Comprehension
Analysis
Critical Thinking
Target 1 2 3 4 5 6 7 8 9 10 11
Description Main Idea Summary/Summarize Infer/ Predict Vocabulary Literary Elements / Text Features Compare / Contrast Cause / Effect Analyze Usefulness of Resources Author’s Purpose Evaluate / Reasoning Drawing Conclusions Extend Beyond the Text
2012 Summary: 6th grade
7th grade
8th grade
Statewide, 6th grade students showed their best performance in: Author’s purpose (target 9) Problem/solution (target 7) Short answer items
Statewide, 7th grade students showed their best performance in: Identify the conflict (target 5) Cause and Effect (target 7)
Statewide, 8th grade students showed their best performance in: Relevant resources (target 8) Compare / Contrast (target 6) Cause and Effect (target 7) Fact and opinion (target 9)
Areas in need of attention are: Summarizing statements (target 2) Relevant resources (target 8) Fact and opinion (target 9) Completion items – fill in the blank
Areas in need of attention are: Literary devices (target 5) Relevant resources (target 8) Draw a conclusion (target 10) Two part short answers: give an inference and support with one detail
Areas in need of attention are: Draw a conclusion (target 10) Analyze using two texts
Data for specific district and school trends within the text-type and strand can be accessed through tools on the State Report Card site under drop down menus. http://reportcard.ospi.k12.wa.us/summary.aspx?year=2010-11. OSPI Reading Assessment 12/19/12
9
High School Students in high school read 5-6 passages balanced between informational and literary genres and answer 40 questions covering the learning strands and targets listed below. The level of rigor for passages has increased above the 6-8 trade level.
Strand Comprehension
Analysis
Critical Thinking
Target 1 2 3 4 5 6 7 8 9 10 11
Description Main Idea Summary/Summarize Infer/ Predict Vocabulary Literary Elements / Text Features Compare / Contrast Cause / Effect Analyze Usefulness of Resources Author’s Purpose Evaluate / Reasoning Drawing Conclusions Extend Beyond the Text
2012 Summary: High School Statewide, high school students showed their best performance in: Identifying tone (target 9) Drawing a conclusion (target 10) Fact and opinion (target 9) Author’s purpose (target 9) Short Answers Areas in need of attention are: Summarizing statements (target 2) Analyzing from two texts Completion items
Data for district and school trends within the text-type and strand can be accessed through tools on the State Report Card site under drop down menus. http://reportcard.ospi.k12.wa.us/summary.aspx?year=2010-11
OSPI Reading Assessment 12/19/12
10
Resources Reading Assessment Website http://www.k12.wa.us/Reading/Assessment/default.aspx Beth Simpson Elementary Reading Assessment Specialist
[email protected]
Cindy Knisely Secondary Reading Assessment Specialist
[email protected]
Terese Otto Reading Assessment Assistant
[email protected]
OSPI Reading Assessment 12/19/12
11
