© 2009 The eLearning Institute. All rights reserved.
Journal of Elementary and Secondary Education
2011 • January • Volume: 2 • Issue: 1
Teachers’ Perceptions of the Effectiveness of Benchmark Assessment Data to Predict Student Math Grades Dr. Lawanna Lewis
ABSTRACT The purpose of this correlational quantitative study was to examine the extent to which teachers perceive the use of benchmark assessment data as effective; the extent to which the time spent teaching mathematics is associated with students’ mathematics grades, and the extent to which the results of math benchmark assessment influence teachers’ pedagogy. Participants were 67 teachers who were administered a 5-point Likert scale assessment. Using descriptive statistics, the results of Research Question 1 showed that teachers agree that frequent use of benchmark assessment data is effective. A Pearson correlation was used with the data to answer Research Question 2 and indicated a positive correlation between instructional time and students’ grades. A significant, positive correlation was found between average class mathematics grades and the number of hours teachers spend teaching mathematics per week, r(65) = .33, p < .01. Students’ class averages ranged from 70.36% - 94.30%. In answer to research Question 3, descriptive statistics revealed that teachers rated benchmark assessment data use as influential to their pedagogy because it helps to determine how they plan for instruction. Based on teacher perceptions, data should be used to plan lessons effectively and to determine specific math skills where students need intervention.
1
Journal of Elementary and Secondary Education
2011 • January • Volume: 2 • Issue: 1
An important area of emphasis in educational research over the past several decades has been the issue of student achievement in mathematics. The improvement of academic achievement and determining methods to narrow the curriculum have served as continuous foci for mathematics reform (Ichilov, 2009; United States Department of Education, 2009). The topic of mathematics reform has motivated researchers to conduct studies that would provide evidence about student performance. The findings from the studies led to the development of strategies designed to maximize the potential of student outcomes and respond to the needs of students in a meaningful way (Ichilov, 2009; Jerald, 2006; United States Department of Education, 2009). These studies, utilizing benchmark assessment data, generated considerable interest in literature. Research indicates few studies have been conducted about the effectiveness of benchmark assessment data with mathematics grades. There are a small number of published accounts of teachers’ perceptions of data use and its influence on teacher pedagogy, teachers’ use of benchmark assessment data to predict students’ mathematics grades, or comparisons between instructional time and students’ mathematics grades. Problem Statement In 1996, the National Mathematics Advisory Council investigated mathematics education for grades Pre-K-12 schools and concluded that the U.S. has not invested in educational research in mathematics. Not everyone agrees on the tools needed to increase student achievement (Bernhardt, 2006). The problem addressed in this study is that teachers remain unconvinced of the effectiveness of benchmark assessment data to predict student math grades. Teachers who opt to use other strategies to improve teaching and learning may ignore data use. Latchat and Smith (2005) studied how four low-performing and high-poverty urban schools used test data to inform school officials of problem areas among their student body. They
2
Journal of Elementary and Secondary Education
2011 • January • Volume: 2 • Issue: 1
found that “teachers are better able to modify instructional strategies when they have current information about the skill levels and proficiencies about their students” (p. 335). Their findings were later supported by Brunner et al. (2006) and Christman et al. (2009). In their study, Latchat and Smith found a link between data use and higher achievement for students when they considered four factors: (a) quality and access, (b) data disaggregation, (c) the role of collaborative inquiry in understanding data, and (d) leadership structures that support data use. Nature of the Study Based on the purpose of the study a quantitative research methodology was used to collect data to answer the research questions. Variables not known about the effectiveness of benchmark assessment data were studied. A correlational measure to investigate any association between instructional time and students’ mathematics grades was determined by use of two factors: (a) teachers’ documented instructional time and (b) logged students’ mathematics grades. A further investigation to examine the extent in which teachers’ use of benchmark assessment data related to teacher pedagogy was determined on a Likert scaled survey—Survey of Instructional Practices Teacher Survey Grades K-8 Mathematics—, which the researcher slightly modified. The target elementary school published data that indicated many schools in the district had deficiencies in the mathematics achievement of students. School officials were seeking ways to improve mathematics scores as well as mathematics instruction. Depka (2006) asserted that teachers should be encouraged to use benchmark assessment data to improve instruction. Suskie and Banta (2009) reported many teachers do not understand how to interpret and use the results of benchmark data to increase student achievement; thus, part of the study was to determine the
3
Journal of Elementary and Secondary Education
2011 • January • Volume: 2 • Issue: 1
extent to which teachers in the target elementary school were actively engaged in using such benchmark data. Purpose Because of the discoveries of The National Council for Teaching Mathematics, both they and the U.S. Department of Education (2008) recommended that more discussion and investigation be conducted. The purpose of this study was to investigate the extent to which teachers agreed and disagreed with the effectiveness of benchmark assessment data used to predict and improve student mathematics grades. More specifically, the study focused on three main objectives: (a) to determine if teachers agree that frequent use of benchmark assessment data is effective in improving students’ mathematics grades, (b) to determine the extent to which the number of hours teachers spend teaching mathematics per week correlate with students’ mathematics grades, and (c) to determine if teachers rate the use of benchmark assessment data as influential in their pedagogy. The first objective was achieved by measuring teachers’ responses to a Likert scale assessment, and by the use of descriptive statistics. The second objective was achieved by using a Pearson correlation to examine any relationship between instructional time data and students’ mathematics grades data. In addition, a Spearman correlation was run on the data to double check the results of the Pearson correlation. The third objective was achieved by the use of descriptive statistics with the data collected from the survey. Sixty-seven teachers participated in the research. The participants taught first through fifth grade students in an urban elementary school in the Southeastern part of the United States. Teaching experience and educational achievement varied among the participants.
4
Journal of Elementary and Secondary Education
2011 • January • Volume: 2 • Issue: 1
Theoretical Framework Instructional psychologists develop theory regarding the best methods to improve learning (Driscoll, 2004). In Gagne’s Theory of Instruction (1985), there are three components: (a) taxonomy of learning outcomes, (b) the conditions of learning, and (c) the nine events of instruction (Bigge & Shermis, 2004). Gagne stated that teachers should consider desired learning outcomes in planning instructional events and the characteristics of learners (Driscoll, 2004). The taxonomy of learning outcomes includes the cognitive domain, which addresses students’ intellectual abilities, students’ attitudes, and motor skills (Gagne, 1985). The conditions of learning additionally address each of these three learning outcomes. The nine events of instruction refer to exterior factors that foster learning: (a) gaining attention, (b) informing learners of the objectives, (c) stimulating recall of prior learning, (d) presenting the stimulus, (e) providing learning guidance, (f) eliciting performance, (g) providing feedback, (h) assessing performance, and (i) enhancing retention and transfer (Gagne, 1985). Gagne (1985) suggested that teachers, particularly mathematics teachers, should plan instruction based on students’ individual needs. Projected learning goals guide the instruction implemented by the teacher along with planned and sequenced events intended to guide the outcomes of student learning. Teachers take actions that will ensure effective learning is validated through instructional events one through five (Kumari & Srivastava, 2005). Tomei (2008) noted that the first event—gaining attention—involves strategies for keeping students on task such as changing stimuli, which is designed to stimulate the learning environment and to keep students focused on the learning. An example is demonstrated through a “hook” or by use of prior knowledge to engage students in the lesson (Kruse, 2004). Teachers provide students with the lesson goals so that they are made aware of outcomes of the lesson (Driscoll, 2004).
5
Journal of Elementary and Secondary Education
2011 • January • Volume: 2 • Issue: 1
The second event informs the learners of objectives, which create a level of expectation for learning, motivates the learners, and prepares them for learning. Gagne (1985) noted that these objectives should be the basis for assessment and to certify that students have mastered the content. The objectives usually begin with a phrase such as “upon completing of this lesson students will be able to…” According to Gagne (1985), objectives activate a process of executive control. Event three—stimulating recall of prerequisites—is used to spiral the curriculum and review previous course material in an effort to help students recall the information when needed (Hays, 2006). Prerequisites allow students to use previously learned information to increase their knowledge base of new information. After re-introducing prior knowledge to students, the fourth event of instruction—presenting the stimulus—can take place. The teacher shares the stimulus based on the goals and objectives of the lesson (Driscoll, 2004). New skills and information are presented and connected to previously learned information (Stolovitch, Pershing, & Keeps, 2006). Event five of instruction—providing learning guidance—promotes guidance for learning by teaching students how to be responsible for their learning (Hays, 2006). Facilitation by teachers allows students to learn by means of discovery as clues and support are offered. Event six through nine correspond to the instruction provided after learning has taken place (Kumari & Srivastava, 2005). Gagne’s sixth event of instruction—eliciting performance—allows the student to share examples that will show evidence of the learning (Driscoll, 2004). Once students have indicated mastery of content through authentic forms of assessments, event seven—providing feedback— takes place. The feedback provides detailed information to indicate students’ weaknesses and strengths (Hays, 2006). In event eight— assessing performance—teachers begin to assess and
6
Journal of Elementary and Secondary Education
2011 • January • Volume: 2 • Issue: 1
promote a sense of student independence (Driscoll, 2006). This can be accomplished through formal and informal assessments such as observations, portfolios, projects, and chapter tests (Driscoll, 2004). The final event of instruction, event nine— enhancing retention and transfer— helps students to retain important information. In this event students are able to transfer the information so that it is applicable to real life situations and to ensure that it is committed to long-term memory (Hays, 2006). Limitations Results of this study provide many useful insights regarding mathematics achievement; however, limitations of the study must be addressed. First, possible threats to the internal validity of the study involve the accuracy of the database, integrity of grade administration, and possible confounds related to differential participant selection. The overrepresentation of females, while providing useful information for this district, may skew the utility of the study outcomes for other populations. Factors such as professional experience, amount of education, the number of years teachers taught mathematics and the familiarity with the school/school district’s curriculum may have influenced many of the unknown variables which affect teaching and learning. Because of the diversity of teachers within the district, schools are encouraged to conduct their own research to understand the influences of demographics specific to their location on benchmark assessment and grades. Second, the study only included elementary teachers, indicating that while teachers reported that data use is useful; the results do not provide evidence for teacher perceptions of students in higher grades. Because only elementary teachers and students’ data were used in the study, the findings cannot be generalized to teachers or students outside of the participants for this study. Third, the study only focused on mathematics and no other subject areas that are
7
Journal of Elementary and Secondary Education
2011 • January • Volume: 2 • Issue: 1
benchmarked, but the information may be useful for school improvement. Data pertaining to other subjects could provide insight to school and district leaders, important information that may be relevant to implement or change the curriculum. Significance of the Study The study provided information that may be useful to teachers in making decisions when using benchmark data to address student deficiencies. The results of the study are especially important in a climate of data-driven school settings, and as school officials are expected to use benchmark data to guide the teaching and learning process. Results may contribute to the literature by providing data for the local school district with which administrators can recommend changes in practices as they relate to the use of benchmark data or instructional time requirements. Results may have policy implications at the school district level that may encourage discussions about the current mathematics curriculum, which may result in policy changes. District leaders might use the results of the study to implement professional development for teachers in an effort to assist them in becoming effective practitioners. State legislators and state education leaders may support further discussions on the topic. Research Questions and Hypotheses Based on the previous section, the following primary research questions guided this quantitative study. The descriptive statistics and hypotheses indicating what the research was to reveal. This supplemented insight into the questions and provided data with which to examine the variables. In this study, the researcher focused on three research questions. 1.
To what extent do teachers perceive the use of benchmark assessment data as effective in improving students’ mathematics grades?
8
Journal of Elementary and Secondary Education
2.
2011 • January • Volume: 2 • Issue: 1
To what extent does the number of hours teachers spend teaching mathematics per week correlate with students’ mathematics grades as measured by the school district’s grading policy? H20: There is no correlation between the number of hours teachers spend teaching mathematics per week and students’ mathematics grades as measured by the school district’s grading policy. H2a: There is a correlation between the number of hours teachers spend teaching mathematics per week and students’ mathematics grades as measured by the school’s district’s grading policy.
3.
To what extent do the results of math benchmark assessment influence teachers’ pedagogy? Literature Review
A review of the literature, pertaining to the relationship between benchmark assessment data and instructional time, shows current research was conducted with findings supporting the benefits of benchmarking to students, teachers, schools, and communities. Benchmark assessment data show areas in which students are struggling and pinpoint areas in curriculum where improvement is needed (Reksten, 2008). Love (2008) asserted that formative assessments can help teachers use the data to determine in what direction teaching and learning needs to go, and how benchmarking provides information not only about student learning but also about teacher effectiveness. In 2001, an amended version of the Elementary and Secondary Education Act (ESEA) of 1965, known as No Child Left Behind (NCLB), was enacted to increase accountability for school districts across the United States, which prompted many school leaders to restructure their curriculum in order to meet the needs of students (Sloan, 2007).
9
Journal of Elementary and Secondary Education
2011 • January • Volume: 2 • Issue: 1
The Importance of Data Driven Instruction Proponents of data-driven instruction believe that teachers should use benchmark data for the purpose of analysis and assessment of problem areas within the curriculum, and with that knowledge adjust classroom instruction (Blink, 2007). Frei (2007) presented “data-driven instruction refers to the process of designing curriculum and instructional strategies to match data from student assessment” (p. 180). California Comprehensive Center and American Institutes for Research (2006) conducted three studies in California schools. The findings from those studies showed that the schools using data to reform the curriculum, and to adjust their program, made significant gains when compared to schools that did not use data for reflection. However, a controversy exists in which some researchers believe that processes must be in place in order for student achievement decision-making to become effective in schools and school districts. Datnow, Park, and Wohlstetter (2007) found that the key strategies for creating processes include (a) data-driven decision-making, (b) establishing a culture of data use and continuous improvement, (c) investing in an information management system, (d) selecting the right data, and (e) building school capacity for data-driven decision making. Their study focused on how teachers used benchmark assessment data, attempting to determine if the use of instructional time affected student grades. To emphasize the impact of data-driven instruction, Datnow, Park, and Kennedy (2007) conducted a qualitative case study of four public high schools in different states in the U.S. The schools they chose were engaging in innovative practices in data-driven instructional decision making (Datnow et al., 2007). Two schools were mid-sized urban high schools and two were smaller charter schools serving low-income minority students. In their study, they analyzed the essentials of the data-driven instructional decision making process in the schools. The
10
Journal of Elementary and Secondary Education
2011 • January • Volume: 2 • Issue: 1
investigators conducted two to three multi-date site visits to each of the high schools and their corresponding district offices. Approximately 50 one-on-one interviews were conducted. Their case studies were composed using the input from more than 90 teachers, administrators, and school system leaders across all four sites. In addition, the investigators conducted two focus groups at each site. Their findings indicated the following; (a) all schools started moving beyond state level assessments and standards to focus on establishing goals that prepared students for college; and (b) all school systems worked together with schools to create coherence among their goals, curriculum, instruction, and assessment. The findings showed the schools established a culture of data use relevant to instructional decision making, and continuous school improvement required leadership at all levels to help teachers make sense of the data. The research participants indicated they all devoted attention, time, and resources to developing infrastructure and methods for data-driven decision making, but each focused on different components. Benchmark Assessments Dell’Olio and Donk (2007) determined that benchmark data provide a more detailed description of areas in which individual children need help, and can assist teachers in structuring learning goals and objectives for classroom instruction. The authors noted that gathering data from benchmark assessments provide teachers a chance to adjust teaching and learning to meet the needs of students more precisely and in a timely fashion. Burns and Gibbons (2008) believed that benchmark assessments are screening tools that monitor the progress of every student, calling specific attention to areas of deficiency. Many educators believe that benchmark assessments should be used as measurement tools to indicate if teachers and students have met their mark. Downey, English, Steffy, and Poston (2008) suggested summative assessments provide meaningful information that teachers
11
Journal of Elementary and Secondary Education
2011 • January • Volume: 2 • Issue: 1
can use. Numerous districts have started to develop tests internally to determine which students have mastered the skills at a particular grade level. Once benchmark assessments have been completed, the next step is to determine which students are performing below expected targets (Burns & Gibbons, 2008). One of the first large scale empirical studies on interim assessments was conducted by Christman et al. (2009) in which they focused on the Philadelphia school district’s use of benchmark assessments. In their study, three key areas were investigated: (a) teachers’ perceptions of the assessments; (b) how teachers used the assessments; and (c) how the emphasis on data-driven teaching affected the effectiveness of exams’ in each of the varying schools. Their study was a multi-method study that drew on three sources for data: (a) student achievement and demographic data from 2005-2007; (b) responses from a district-wide teacher survey; and (c) in-depth, qualitative research in 10 schools from Fall 2005 to Spring 2007. The findings indicated that the benchmarks established clear expectations pertaining to (a) what teachers should teach and (b) at what pace they should teach. Educational Accountability Educational accountability, a central theme for improving education, became the centerpiece for President Bush’s agenda in 2001. Educational accountability included the requirement that all students in grades three through eight be tested in reading and mathematics each year. This premise was based on the beliefs of stakeholders who found educational achievement inadequate (Ryan & Shepard, 2008). Subotnik and Walberg (2006) noted that a time when legislative and public demands called for stronger accountability and continuous education reform in an effort to increase student achievement, school officials realized that the amount of instruction that students receive affects their development. The authors further noted that U.S. students spend only 8.2% of their time in school during the first 18 years of their lives,
12
Journal of Elementary and Secondary Education
2011 • January • Volume: 2 • Issue: 1
and the child’s parents influence a third of the child’s 18 years, which has an effect on children’s intellectual development and their ability to process information. In a study done by Entwisle and Alexander (1992) in Baltimore on poor African American and White students, both groups lost achievement during the summer months—during summer vacation. Summer vacation may be particularly disadvantageous to students who live in poverty because those students are not exposed to academics. Sunderman (2007) stated that a sound educational accountability system employs indicators that encourage constructive educational practices that will generate trustworthy information about the status and growth of learning and teaching practices. Patton (2008) argued performance monitoring is an indicator that will assess the impact of government policies on services and identify well-performing or underperforming schools. Alba and Nee (2003) pointed out that teachers and school leaders are accountable to the public in many states, and parents can compare schools demographically and see the extent to which the achievement gap closes. The authors also argued that in a changing world in which societal need dictates educational reform, there is a need to place greater emphasis on students experiencing success in school. The Importance of Increased Instructional Time Because many think that time in school is a factor, appropriate use of time is critical to improve student learning. U.S. students spend the shortest amount time in school compared to other countries in the industrialized world, do less homework than students from other countries, and are often influenced by out-of-school factors (Subotnik & Walberg, 2006). Stallings (1979) and Walberg (1979) have argued that the use of time is more important than time itself. Reynolds and Fletcher-Janzen (2007) noted that research on effective teachers indicated that it is best to plan the use of the school day and teach in small groups so that students can obtain more
13
Journal of Elementary and Secondary Education
2011 • January • Volume: 2 • Issue: 1
instructional time. Salkind and Rasmussen (2008) listed responses by teachers on surveys sharing ideas about testing, and one response stated that teachers feel the need to reallocate and increase instructional time so that more time is allowed to focus on tested material. Downey et al. (2008) stated that instructional time is very important and should be used wisely. Teachers can allocate instructional time appropriately by implementing strategies that help teachers provide instructional support, based on the needs of the students (Rathvon, 2008). Once teachers have allocated instructional time to accommodate students according to priorities, teachers must be willing to defend that time (Glatthorn & Jailall, 2008). According to Reynolds and Fletcher-Janzen (2008), when teachers use data to help them to make decisions about the types of interventions students may need to address their deficits, it allows more instructional time to teach the basic skills that students lack. Reynolds and Fletcher-Janzen noted that instructional time is highly related to academic. Walberg (1982) studied instructional time and achievement, and found correlation ranges from .13 to .71 with a median of .41. Out of the 25 studies reviewed as time on learning, 95% reported a positive correlation. The National Council of Teachers for Mathematics (2006) suggest at least one hour of mathematics instruction each day and believe that this gives students 50% more time with mathematics whereby students will receive 180 hours of instruction a year. The Project Massachussetts 2020 did an investigation on how students spend their school day. Data analysis consisted of time diaries from a national sample of elementary school teachers, a survey, and a time diary in which teachers were asked to keep precise records of how they spent their time on one randomly designated school day. The results indicated that on a typical school day, teachers spend time on an average of 14 discrete activities: 64.4% are academic, 14.6% are maintenance, 11.9% are enrichment, and 6.8% are recess-related.
14
Journal of Elementary and Secondary Education
2011 • January • Volume: 2 • Issue: 1
How Teachers and Administrators Use Data Teachers in various ways use data analysis, but all is done to strengthen teaching and learning. Teachers most often use assessment data to disaggregate the results that provide information about subgroups within their classes so that decisions can be made about planning and instruction (Depka, 2006). Mclaughlin and Talbert (2006) argued that this allows teachers to look for trends in the students’ performances to encourage training for teachers and administrators. Item analysis allows teachers to view the items that students miss to determine what area of the mathematics curriculum needs attention, and to determine the next steps for teaching (Stiles, Mundry, & DiRanna, 2008). Wiburg and Brown (2006) stated that professional development activities are chosen to accommodate teachers’ needs based on assessment results and close the gap between the school or district’s goals and student performance. The results of the study indicated that although the schools approached the process in which data were used differently, the schools shared some commonalities. Further, school officials were convinced that the system for using data plays an important role in helping school officials understand how to use data. Processes must be in place in order for the system to take shape and for decision-making relating to student achievement to become effective in schools and school districts. Teacher Perceptions on the Usefulness of Benchmark There has been minimal number of recent studies on data assessment. Because of legislative mandates to increase school and teacher accountability, researchers have started to focus on ways to address the achievement gaps that exist among students such as a project sponsored by the Center for Rural Pennsylvania to investigate the mandates of NCLB (Smeaton & Waters, 2008). Studies on teacher use of assessment data have focused on teachers using
15
Journal of Elementary and Secondary Education
2011 • January • Volume: 2 • Issue: 1
assessment data to understand the diverse population of students and their needs; however, research findings link continuous benchmark assessments, tracking student and school progress, with an increase in student achievement. In a Long Island study, DeLuca, Dieringer, Nasta, Peters, and Mann (2007) examined teachers’ perceptions of their own practices regarding data-driven decision-making and their perceptions of their elementary school principals’ professional practices. The study focused on five elementary schools in Long Island, NY in which 118 teachers completed the Principal and Teacher Learning Community Scale. A paired samples t test was used. The results indicated teachers’ perceptions of their own implementation of data-driven decision-making exceeded their view of principals’ implementation of data-driven decision-making. Research Methods and Design The study was conducted following a quantitative study design involving survey research and descriptive data analysis detailing the extent of teachers’ use of benchmark assessment data, instructional time, and the extent of influence of benchmark assessment data on teacher pedagogy. A Pearson correlation coefficient was also used for data analysis. Data were collected on teacher responses through the Benchmark Assessment Data Survey, which was created by Wisconsin Center for Education Research (2004). Gall et al. (2006) stated that questionnaires are often used in quantitative research because of consistency and compatibility. Research Question 1 explored the extent to which teachers agreed that frequent use of benchmark assessment data was effective in improving students’ mathematics grades. This was first addressed by presenting descriptive statistics (i.e., medians, means, standard deviations, frequencies and percentages). Research Question 2 examined the relationship between the number of hours teachers spent teaching mathematics per
16
Journal of Elementary and Secondary Education
2011 • January • Volume: 2 • Issue: 1
week (instructional time), and teachers’ class average as measured by the school district’s grading policy, which was addressed via a Pearson correlation. For Research Question 3, the researcher investigated whether teachers rated mathematics benchmark assessment data as influential of their pedagogy and used descriptive statistics to describe the teachers’ responses. The eight constructs that supported the research questions were achievement gap, benchmark, curriculum, educational accountability, instructional time, intervention strategies, and subgroup. Participants Approval was obtained from the principal of the study site to gain permission to survey teachers. A convenience sample was used; all the participants worked at the study site (Creswell, 2006). One hundred first grade through fifth grade teachers were asked to voluntarily participated in the study. The permission to use teacher data was granted by the local school district. This design was economical, participating teachers could be utilized immediately and the statistical power analysis revealed that the number of teachers needed were available at the site. Sixty-seven grade one though five teachers participated in the study. Teachers were required to teach the mandated time allotted for mathematics, but increased mathematics instructional time was provided to students based on their needs. To obtain a desired confidence level of 95% (using a confidence interval of 5) a sample size of 64 would have been required (Israel, 2009); 67 teachers were selected to participate in the study. Only certified teachers at the local study site were asked to volunteer for the study. The teachers were representative of teachers in the school, which includes regular education, inclusion, and special education classes. The teachers’ professional experience ranged from 1 to 25 years. The teachers were familiar with benchmark assessments and instructional time. All teacher participants taught self-contained or inclusion classes or they worked collaboratively
17
Journal of Elementary and Secondary Education
2011 • January • Volume: 2 • Issue: 1
with a resource teacher in a regular classroom setting. The participants were familiar with benchmark assessments in that they administered the benchmark assessment tests three times per year. Data Collection, Processing, and Analysis The primary methods of data collection in this study included (a) a survey assessing the influence of benchmark assessment data, (b) teacher use of instructional time, and (c) student grades. Data were entered into Statistical Package for the Social Sciences (SPSS 15.0) computer software and analyzed statistically. A Pearson correlation was conducted to determine the correlation between instructional time data and the students’ grades. A Spearman correlation was conducted to validate the findings of the Pearson correlation. Survey. The survey was distributed to 67 certified classroom teachers at the elementary school study site who had been at the school since at least August 2009. The survey packets included written instructions and an informed consent form. Teachers completed the survey individually and were informed that surveys were anonymous and did not contain teacheridentifying data. The surveys and informed consent form were collected separately to ensure confidentiality of survey data. The data were analyzed statistically. Instructional time. Teachers selected the average time that they spent teaching mathematics to students daily from the following list: (a) 30 to 40 minutes, (b) 41 to 50 minutes, (c) 51 to 60 minutes, (d) 61 to 90 minutes, and (e) 91 to 120 minutes. Each teacher was required to teach mathematics for 60 minutes of allocated time, mandated by the state of Georgia, but had the option of allocating 30 additional minutes, which is granted by local school officials. For reasons unknown to the researcher, some teachers chose less time, and some teachers chose more time. Other teachers only taught the required number of minutes that are allocated by the Georgia
18
Journal of Elementary and Secondary Education
2011 • January • Volume: 2 • Issue: 1
Department of Education. Schools within the district have an option to increase the time to accommodate student needs. The school study site had chosen to use the option of up to 120 minutes. Sixty-seven teachers returned recorded documentation of instructional time for the 9week period of the study. The instructional time was displayed in a Microsoft Excel spreadsheet. Student grades. Grades were operationalized by converting student grades as measured by the school district’s grading policy, to end-of-the-nine week’s class mathematics averages in 2009. Participating teachers provided end-of-the-nine weeks mathematics classroom averages for all students. These grades represented class averages based on students who passed or failed mathematics. The grades were coded on a 0-100 scale as determined by the district. After all student grades in each class had been recorded, the points for the entire class were added producing a total score. The total score was divided by the number of students in the class to produce a mean. A computation for the standard deviation described the average distance between the values and the mean. Ethical Assurances The following ethical considerations were addressed in the study: (a) confidentiality, (b) informed consent, (c) minimal risk to participants, and (d) utilizing instruments that measured what was intended to be measured. The researcher informed potential participants about the study and gained their consent. The informed consent letter provided enough information that each participant could make a decision about whether to participate or not. It was important to inform participants of their rights, because some might have felt obligated once the principal or district officials agreed to allow the study (Hatch, 2002). Participants were informed that they could withdraw at any time without penalty and that their right to participate was on a voluntary basis.
19
Journal of Elementary and Secondary Education
2011 • January • Volume: 2 • Issue: 1
Confidentiality in the collection and use of data was handled with extreme caution because sensitive information was at hand. The school principal as well as district officials gave permission to the researcher to have access to student data. However, eventually primary grading data reaches the public domain. Student grades affect the district when entered into the public domain in response to mandates by the NCLB. Nevertheless, caution was taken to protect the identity of both participants and the subject school, and information pertinent to the study will be kept in a secure location for 3 years, and then it will be shredded and discarded. Evaluation of Findings The researcher attempted to examine the extent to which teachers agreed or disagreed about the effectiveness of benchmark assessment factors for improving academic achievement. To achieve this, research data was collected from 67 elementary school teachers and statistical analyses were implemented to examine the extent to which teachers agree or disagree with the effectiveness of benchmark assessment data. The quantitative data collection method was a survey, instructional time logs, and teacher reports. One theory that was grounded in the research was based on Gagne’s Theory of Instruction (1985). This theory was manifested in the teacher reports about their perception on benchmark assessment data and connects to the research questions. Correlated instructional time, teachers’ mathematics class averages, and teachers’ perceptions of benchmark assessments in reference to pedagogical changes were assessed. These results provide support for ongoing efforts to increase student achievement in mathematics. Specifically, the study results showed that (a) teachers use benchmark assessment data to improve students’ mathematics grades, (b) there is a significant positive relationship between
20
Journal of Elementary and Secondary Education
2011 • January • Volume: 2 • Issue: 1
instructional time and student grades, and (c) most teachers perceive benchmark assessment data as being useful in their pedagogy. There are similar findings between this benchmark data study and other studies on the effectiveness of benchmark assessment data and instructional time. The findings of Research Question 2 are likened to the studies completed by Walberg (1982), which indicated a positive correlation involving instructional time and achievement. The findings of Research Question 3 are similar to the study conducted by Christman et al. (2009) where the survey results indicated that teachers felt positive towards the usefulness of benchmarks. Christman et al. (2009) conducted an examination of Philadelphia School Districts’ use of benchmark assessments. The findings of this current study are similar with those of the study conducted by Christman et al. The findings of both studies indicate that the teachers felt positively toward the usefulness of benchmarks. The current study’s findings also support previous research emphasizing the impact of data driven instruction by Datnow, Park, and Kennedy (2007), who implemented a qualitative case study of four public high schools in different states in the U.S. to analyze the essentials of the data-driven instructional decision making process in schools. The Datnow et al. study suggested schools that use data to make instructional decisions report improved scores, which was supported by the current study’s findings that students’ mathematics grades improved most in the classes where teachers reported benchmark assessment data use. The statistically significant correlation between instructional time and students’ students’ mathematics grades aligned with previous studies. For example, Downey et al. (2008) stated that the strongest relationship is between instructional time and student achievement when teachers align the curriculum at the right level of difficulty for the student. This researcher indicated that
21
Journal of Elementary and Secondary Education
2011 • January • Volume: 2 • Issue: 1
the class mathematics average in this current study was 83%. This finding shows a significant positive correlation between instructional time and teachers’ mathematics class average. Downey et al. (2008) acknowledged that teachers must provide differentiation in time for low performing students, but that the curriculum must be aligned with local and state objectives and ongoing data use to effectively meet the needs of all students. A fourth research study that correlates with this study was investigated by Gettinger (1985) that investigated the extent to which spending less time on academics affected students overall achievement. The results indicated that when less instructional time is afforded to students, there is a direct negative effect on student achievement. The study included 171 fourth and fifth grade students. Initial learning and student retention dropped significantly when students spent less time on task than time needed to earn the task. Similarly, this study was based on the results of less instructional time versus more instructional time provided to students. The evaluation of the findings of each research question is as follows: Question 1. To what extent do teachers perceive the use of benchmark assessment data is effective in improving students’ mathematics grades? Sixty-four percent of the teacher participants reportedly used benchmark assessment data frequently. A concern that many participants indicated was that they did not trust the data to be accurate. With over 50% of the participants reporting that they viewed data as a helpful tool, it was validated by both the responses on the survey and the grade reports provided by the teachers. Question 2. To what extent does the number of hours teachers spend teaching mathematics per week correlate with students’ mathematics grades as measured by the school district’s grading policy? Forty-two percent of the teachers taught math 51-60 minutes a day for five days a week. Students’ mathematics quarter averages ranged between 70.36 - 94.30% and
22
Journal of Elementary and Secondary Education
2011 • January • Volume: 2 • Issue: 1
the average class grade was 83.11. The minimum time reported by participants indicated a class average of 70.36% while the maximum time reported indicated a class average of 94.30%. Question 3. To what extent do the results of math benchmark assessment influence teachers’ pedagogy? Teachers reported “preparation of students” influences teacher as the highest average rating at 3.87 followed by “increased instructional time” with a mean of 3.37. Teachers reported the “district test” influence teaching with the lowest rating with an average of 2.00. The results of this study indicated that teachers reported to have used data to plan effectively, diagnose student learning, and accommodate student needs, and as a result, student mathematics grades increased. The findings of the benchmark data use study indicated that teachers who reportedly increased instructional time also had an increase in student grades. The more time that is spent in an area where a student is struggling, the more the student will master the skills. The important implication of this finding is that this information can be used by local school district officials to modify the time students spend in mathematics to reflect more minutes of mathematics instruction time, thus leading to a greater potential for increased student achievement. State legislators may use the findings of the current study to increase the minimum amount of time students spend in mathematics. By increasing the time, students will have more time to become familiar with mathematics strategies. The current results also point out that data use will improve student mathematics grades. School leaders will be able to share data with students, teachers, and the community so that decisions can be made about the teaching and learning students receive. The data can be disaggregated to determine which students are performing the lowest on formative assessments,
23
Journal of Elementary and Secondary Education
2011 • January • Volume: 2 • Issue: 1
as well as provide summative assessments. This can help schools to modify the curriculum and to implement new programs as the need arises. To fulfill the purpose of this study, it was necessary to use a survey instrument to measure teachers’ use of benchmark data, the amount of instructional time spent in mathematics, and teacher perceptions of benchmark assessments. The survey was created through the modification of an existing survey instrument; Survey of Instructional Practices Teacher Survey Grades K-8 Mathematics developed by the Wisconsin Center for Education Research (2004). Conclusions Benchmark assessment data are useful if they are used for the purpose intended: student achievement. It is a means to pinpoint areas of the curriculum that may need more attention either with individual students or with the entire class. It further allows teachers to assess areas where they struggle and need extra support. Data use forces a collaborative approach that will provide teachers more time to discuss and seek other ways to increase their knowledge base of a certain skill or approach. The benchmark assessments can be useful for monitoring progress toward students’ performance. Teachers who spend more time to address the deficiencies will likely see an increase in student grades. The analysis in this study showed that 64% of the teachers reported the use of benchmark assessment data frequently, which points to a positive trend of benchmark assessment use and potentially better outcomes in students’ mathematics grades. This study indicates that both teachers use of benchmark assessment data and increase in instructional time fill an important role in the success of students. Recommendations Recommendations for practice. This study demonstrated the extent to which teachers reported benchmark assessment data useful. It was an analysis of the relationship between
24
Journal of Elementary and Secondary Education
2011 • January • Volume: 2 • Issue: 1
instructional time and teacher mathematics class averages, and was a description of how teachers perceive the effect of benchmark assessment data on pedagogy. The results of the study have particular implications for schools and districts because it provides educators ways in which to use assessment data to improve teaching and learning in a way that will benefit both the consumers and the community. Educators might use the results of this study as an impetus for collecting data to guide instructional decisions and maximize instructional time to increase student achievement. Educators might also examine the way instructional time is used particularly in math. School administrators might use the data as an historical approach to look for trends that students in certain subgroups may fall under, thus providing more support to those students. The findings of this study may help school officials further investigate students who are experiencing difficulty in mathematics and potentially find the causes. The outcomes of the current research also provide information that may be used to implement professional development in areas that are critical to student success and teacher productivity. The findings of this study may aid schools in the allocation of funds for professional learning that may do very little in the area of student achievement. Implications The intent of this study was to investigate teachers perceptions of benchmark assessment data use to predict student math grades using a quantitative method. Analysis of data collected revealed the following for each research question: The study contributes to the literature for benchmark assessment data use and formative assessments in several ways. The majority of the participants reported the use of the data to plan lessons effectively, modify instruction, address students’ needs, and spend more time teaching mathematics. The results from the study indicated
25
Journal of Elementary and Secondary Education
2011 • January • Volume: 2 • Issue: 1
that teachers perceived that benchmark assessment data as effective when it is used to improve achievement.
26
Journal of Elementary and Secondary Education
2011 • January • Volume: 2 • Issue: 1
References Alakeson, V. (2004). A 2020 vision for early years: Extending choice, improving life chances. London: Social Market Foundation. Alba, R., & Nee, V. (2003). Remaking the American mainstream: Assimilation and contemporary immigration. Cambridge, MA: Harvard University Press. Barr, R. D., & Parrett, W. (2007). Saving our students, saving our schools: 50 proven strategies for helping underachieving students and improving schools. Thousand Oaks, CA: Corwin Press. Bernhardt, V. L. (2006). Using data to improve student learning in school districts. Larchmont, NY: Eye on Education. Blink, R. (2007). Data-driven instructional leadership. Larchmont, NY: Eye on Education. Burns, M. K., & Gibbons, K. (2008). Implementing response-to-intervention in elementary and secondary schools. Abingdon, England: CRC Press. Christman, J. B., Neild, R. C., Buckley, K., Blanc, S., Liu, R., Mitchell, C., & Travers, E. (2009). Making the most of interim assessment data: Lessons from Philadelphia. Research for action. Retrieved July 22, 2009, from http://pdf.researchforaction.org/rfapdf/publication/pdf_file/558/Christman_J_Making_the _Most_of_Interim_Assessment_Data.pdf Cortiella, C. (2008). Supplemental education services under no child left behind. Great schools. Involved parents. Successful kids. Retrieved December 12, 2008, from http://www.greatschools.org/special-education/legal-rights/supplemental-educationalservices-under-no-child-left-behind.gs?content=871 Darling-Hammond, L. D., & Brandsford, J. (2005). Preparing teachers for a changing world. Hoboken, NJ: John Wiley & Sons. Datnow, A., Park, V., & Wohlstetter, P. (2007). Achieving with data: How high-performing school systems use data to improve instruction for elementary students. Los Angeles: Center on Educational Governance. DeLuca, D., Dieringer, E., Nasta, M., Peters, G., & Mann, D. (2007). Implementing data driven decision making: Teachers' perceptions of their own practices and their principals' practices. Paper presented at the annual meeting of the American Association of Colleges for Teacher Education, New York, NY. Retrieved July 17, 2010, from http://www.allacademic.com//meta/p_mla_apa_research_citation/1/4/1/9/5/pages141950/ p141950-1.php Depka, E. (2006). The data guidebook for teachers and leaders. Thousand Oaks, CA: Corwin Press.
27
Journal of Elementary and Secondary Education
2011 • January • Volume: 2 • Issue: 1
Downey, C. J., English, F. W., Steffy, B. E., & Poston, W. K. (2008). 50 ways to close the achievement gap. Thousand Oaks, CA: Corwin Press. Driscoll, M. P. (2004). Psychology of learning for instruction (3rd ed.). Boston, MA: Pearson. Easton, L. B. (2007). Engaging the disengaged: How schools can help struggling students succeed. Thousand Oaks, CA: Corwin Press. Elliot, J. L., & Thurlow, M. L. (2006). Improving test performance of students with disabilities. Thousand Oaks, CA: Corwin Press. Entwisle, D. R., & Alexander, K. L. (1992). Entry into schools: The beginning school transition and educational stratification in the United States. Annual Review of Sociology, 19, 401423. Frei, S. (2008). Teaching mathematics today. Huntington Beach, CA: Shell Education. Gagne, R. (1985). Gagne's Nine Events of Instruction. Retrieved on November 15, 2009 from http://online.sfsu.edu/~foreman/itec800/finalprojects/annie/gagne'snineevents.html Glatthorn, A. A., & Jailall, J. M. (2008). The principal as curriculum leader: Shaping what is taught and tested. Thousand Oaks, CA: Corwin Press. Ichilov, O. (2009). The retreat from public education. New York: Springer. Hall, T. (2002). Differentiated instruction. Wakefield, MA: National Center on Accessing the General Curriculum. Retrieved November 20, 2007 from http://www.cast.org/publications/ncac/ncac_diffinstruc.html Hays, R. T. (2006). The science of learning: A systems theory perspective. Boca Raton, FL: BrownWalker Press. Henderson, S. (2008). Do Benchmark assessments make a difference? A first look. R & D Alert, 43(2). Retrieved September 17, 2009, from http://www.wested.org/cs/we/view/feat/159 Holcomb, E. L. (2004). Getting excited about data. Thousand Oaks: Corwin Press. Kimmelman, P., & Doyle, D. P. (2006). Implementing NCLB. Thousand Oaks, CA: Corwin Press. Kumari, S., & Srivastava, D. S. (2005). Curriculum and instruction. Adarsha Nagar, India: Delhi Books. Lachat, M. A., & Smith, S. (2005). Practices that support data use in urban high schools. Journal of Education for Students Placed At Risk, 10(3), 333-349. McLaughlin, M. W., & Talbert, J. E. (2006). Building school-based teacher learning communities. New York: Teacher College Press.
28
Journal of Elementary and Secondary Education
2011 • January • Volume: 2 • Issue: 1
Mundry, S., & DiRanna, K. (2008). The data coach’s guide to improving learning for all students: Unleashing the power of collaborative inquiry. Thousand Oaks, CA: Corwin Press. Olson, L. (2005, November). Benchmark assessments offer regular checkups on student achievement. Education Week 25(13), 13-14. Patton, M. Q. (2008). Utilizaton-focused evaluation. Thousand Oaks, CA: Sage. Pete, B. M., & Fogarty, R. (2005). Close the achievement gap: Simple strategies that work. Thousand Oaks, CA: Corwin Press. Rathvon, N. (2008). Effective school interventions: Evidence-based strategies for improving student outcomes. New York: Guilford Press. Rotatori, A. F., Obiakor, F. E., & Burkhardt, S. (2007). Current perspectives in special education administration. Bingley, England: Emerald Group. Ryan,K. E., & Shepard, L. A. (2008). The future of test-based educational accountability. Florence, KY: Taylor & Francis. Salkind, N. J., & Rasmussen, K. (2008). Encyclopedia of educational psychology. Thousand Oaks, CA: Sage. Sherman, W. H. (2008). Educational policy. Thousand Oaks, CA: Corwin Press. Smeaton, P., & Waters, F. (2008). A statewide investigation into meeting the mandates of no child left behind. The Center for Rural Pennsylvania. Retrieved August 2, 2009, from http://www.rural.palegislature.us/NCLB.pdf Stallings, J., Needles, M., & Strayrook, N. (1979). The teaching of basic reading skills in secondary schools, phase I and phase II. Menlo Park, CA: Standford Research Institute. Subotnik, R. F., & Walberg, H. J. (2006). Research in educational productivity. Charlotte, NC: Information Age. Sunderman, G. (2008). Holding NCLB accountable. Thousand Oaks, CA: Corwin Press. Superfine, B.M. (2008). The courts and standards-based education reform. New York: Oxford University Press. Suskie, L., & Banta, T.W. (2009). Assessing student learning. Hoboken, NJ: John Wiley. Tomlinson, C. (2001). How to Differentiate Instruction in Mixed-Ability Classrooms (2nd ed.). Alexandria, VA: Association for Supervision and Curriculum Development. Tomlinson, C. A. (2002, November 6). Proficiency is not enough. Education Week, 22, 36, 38.
29
Journal of Elementary and Secondary Education
2011 • January • Volume: 2 • Issue: 1
Tomlinson, C. A., & Strickland, C. (2005). Differentiation in practice: A resource guide for differentiating curriculum. Alexandria, VA: Association for Supervision and Curriculum Development. Tough, P. (2006, November). What it takes to make a student. New York Times Magazine. Retrieved January 21, 2009, from Virginia Board of Education. (2008). First-year warned principals' institute: SOA requirements for school improvement plans and October 1 reports [PowerPoint slides]. Retrieved on July 17, 2010, from http://www.slideshare.net/Annie05/soa-requirements-for-school-improvement-planspresentation United States Department of Education (2004). The No Child Left Behind Act. Washington: United Government Printing Office. Walberg, H. J. (1982). What makes schooling effective? A synthesis and a critique of three national studies. Contemporary Education: A Journal of Reviews, 1(1), 22-34. Webb, N. M. (1984). Sex differences in interaction and achievement in cooperative small groups. Journal of Educational Psychology, 36(1), 33-4. Wilson, L. W. (2007). Improving your elementary school. Larchmont, NY: Eye on Education.
30
Journal of Elementary and Secondary Education
2011 • January • Volume: 2 • Issue: 1
Author Information: Dr. Lawanna Lewis has 14 years of teaching experience as a public school educator. In addition, she is an adjunct instructor at Grand Canyon University. She is the co-founder and CEO of iEducate Consultants, which is a consultant firm that offers professional learning opportunities for teachers and administrators. Lawanna has presented numerous professional development seminars at the Central Alabama Regional Inservice Center (Alabama State University) and in other school districts.
31
Journal of Elementary and Secondary Education
2011 • January • Volume: 2 • Issue: 1
32
Journal of Elementary and Secondary Education
2011 • January • Volume: 2 • Issue: 1
Author Information:
33
