Peer Evaluation as an Active Learning Technique

Peer Evaluation as an Active Learning Technique Nancy Reese-Durham An action research was carried out during the fall 2003 term. Nineteen students in ...
Author: Elinor Houston
0 downloads 1 Views 522KB Size
Peer Evaluation as an Active Learning Technique Nancy Reese-Durham An action research was carried out during the fall 2003 term. Nineteen students in an Applied Research in Education course at Fayettevilie State University, North Carolina were participants in the study to determine the impact of using peers in the evaluation of a partial research paper. The answers to three questions were sought: I)To what extent is peer feedback meaningful and effective; 2) What lessons (if any) do the peer-evaluators learn from the activity; and 3) To what extent does the peer evaluation process result in better research papers for student researchers and peer evaluators? Results showed that all ofthe student researchers agreed that the feedback was helpful, constructive, clear and understandable. Further, results indicated that the student researchers realized that they need to include more substantial information in the review ofthe literature section and that the activity was helpful in the final paper revision process. Overall, the quality ofthe papers from this class was significantly higher than papers collected from previous classes.

In my introductory research course the focus is more on getting my students to learn the process than getting them to master the process. Learning the terminology, being competent in statistical computations, understanding how to analyze research and other basic components of educational research are essential knowledge for the student. Evaluation is a crucial part ofthe learning process (Barrett, 1986), Consequently, feedback and evaluation on a student's progress are important both to the student and to the faculty. Students need information on their progress to make improvements in their work. Although a faculty evaluation of student work is perhaps the most common type of evaluation, the use of peers in the evaluation process may be just as effective in their academic growth. Previous evaluations of students' work involved me, as the instructor, in reviewing

Nancy Reese-Durham, Ph. D., Department of Middle Grades, Secondary and Special Education, School of Education, Fayettevilie State University. Correspondence concerning this article should be addressed to Dr. Nancy Reese-Durham, 1200 Murchison Road, Fayettevilie, NC 28301; Email: [email protected].

drafts of papers, inserting comments in red ink, and returning the papers to the students. In some Instances, students expressed how the editing oftheir papers was intimidating and damaging to their self-esteem. Discovering another way to effectively evaluate student research while at the same time making the evaluation process a learning activity would eliminate the problems associated with editing, such as the loss in self-esteem and increase the involvement ofthe class in the process. Peer Evaluation According to Boyd (1989), evaluation can be used to "provide constructive criticism and suggestions to improve weak areas and amplify strengths." (p,2). Two types of evaluation exist - formative and summative. Formative evaluation involves giving feedback that is focused on changing processes as they are happening. On the other hand, summative evaluation occurs at the end of the process and focuses on judging the quality of the outcome (Boulmetis and Dutwin, 2000). Using peers has been found to be an effective addition to the evaluation process. Formative peer evaluations allow peers to work collaboratively to assess each others'

338

Peer Evaluation . . 1339

work and assist one another in efforts to that is required in a research course. There strengthen research. Working with one's is an ardent amount of information for the peers allows interchange of ideas and meth- beginning students to understand in order to odologies resulting in a more refined product "do" research. For example, basic research (Powell, 1992;Sapin-Piane, 1993). Studies terminology terms (e.g., independent and by Saavedra and Kwun (199.3) found that "on dependent variables, internal and external the whole, both field and laboratory studies validity, hypothesis, types of research and indicate that peer assessment is a valid and types of experimental design) which represent reliable evaluation procedure." (p. 450) basic concepts which must be understood by Lisk (2000) argued that this process of beginning researchers. cooperative learning must include a number Lastly, students are faced with a dilemma of essential conditions to be successful in when attempting to conduct research in their learning environments. The conditions in- own classrooms. This aspect of applying clude: (a) a clear set of learning objectives research skills acquired in the classroom is that are accepted by all students, (b) positive vital to "learning how to conduct research". interdependence, (c) positive social interac- However, if a student chooses to use curtion behaviorand attitudes, and (d) individual rent classroom test data (i.e., pre and post accountability. test scores) as a measuring instrument, the Portfolio assessment, self-assessment student may need to "create" post-test scores and peer-review are forms of assessment that if the college course ends before the actual encourage students to engage continuously post-test scores are available. and foster a deeper approach to learning. Key To address these issues and improve the elements of these approaches are reflection, instructional process, I sought toexamine the feedback, and integration of learning and as- impact of using peers in the evaluation of a sessment (Dochy and McDowell, 1997). partial research paper. Specifically, I wanted to answer the following questions: Peer Evaluation of Research Papers 1. To what extent is peer feedback meaningThere are several issues that are assoful and effective? ciated with evaluating beginning research 2. What lessons (if any) do the peer-evalustudents. Beginning research students are not ators learn from the activity? only required to learn about the process of 3. To what extent does the peer evaluation "doing" but are also at the same time applying process result in better research papers the process to a real classroom situation. for student researchers and peer evaluOne issue is the wide range of research ators? skill that students bring to the class. A graduate education program generally require a In this study, the students evaluated only research class as a core course in the cur- thefirstpart oftheirclassmates research paper. riculum and is viewed as a course which This "partial" section inluded the Introducwill provide a foundation for future research tion, Statement of the Problem, Review of projects which students will have to complete the Literature, Statement of the Hypothesis in their programs. While some students will and the Methods section. The term "peer have completed a research paper or project evaluation" is defined as "a participation of in their undergraduate programs, for most colleagues in the development and assessment students the graduate course will likely be of activities" (Benshoff, 1988). Colleagues their first exposure to the research process. can helpeach other by reviewing each other's Another issue is the number of new work, and sometimes, simply sitting and concepts and the amount of information talking with each other about their studies is

3401 Journal of Instructional Psyctiology, Vol. 32, No. 4

meaningful. In this way, critical and supportive feedback is emphasized while evaluation is de-emphasized. Methodology This project involved the use of one graduate educational research class. The subjects in the course were nineteen (19) teachers who, when asked to rate their skills at conducting research, showed a range of scores from 3 to 10 with 6 being the median score and 5 being the mode score (see Table 1). Most students expressed that they knew there was much more to learn about conducting research. Table 1 Self Report of Research Skill on a Scale of I-JO, with 10 ^high Number 1 ?. 3 4

6 7 8 9 0 1 7 13 14 15 16 7 8 9

Median Mode

Research Skill 6.5 4

7 7 7 6 5 5 5 5 10 5 3 4

6

45

6.5 6 5

An educational research text by Gay and Airasian (2000) served as a guide for the instruments designed for this study. Prior to the beginning ofthe activity, rules forcompleting the peer evaluation forms were shared and discussed with the class. Each student was given forms to share with two peers of his/her choice in the evaluation ofthe research paper. Each student provided a copy ofthe partial research paper to the two self-selected classmates to review and submit suggestions and/or comments

about the paper. Each reviewer signed the review sheet to indicate who had reviewed the research paper for the researcher. In this way, the researchercouldaskforclarification from the reviewer about suggested revisions prior to making final revisions to the paper. The evaluation form was divided into sections that corresponded to the portions of the paper that were to have been completed at the midpoint ofthe course. Specific questions about the problem statement, review of related literature, hypothesis and descriptions of the participants were included on the review form. For example, a question about the "Review of the Related Literature" section asked the following: Is the review comprehensive? Although, the peer consultant had not reviewed the information for the topic, it was expected that the written information provided by the student researcher would be comprehensive enough for the reviewer to understand the topic. Since each student in the course had completed similar research papers each reviewer was knowledgeable about the type of information that was to be included in each section of the paper. A column on the evaluation form was included for the peer reviewer to make comments relative to each question. After the forms were returned, the student researcher reviewed the comments and talked with each reviewer concerning comments needing clarifications. The student researcher then composed a paragraph about the lessons learned from the activity and rated the activity on a 5-point Likert scale (l=strongly disagree to 5=strongly agree) on four statements relative to the activity. Lastly, a questionnaire was developed for the peer evaluators to make comments about the activity. This learning activity was administered during the mid-term examination week, approximately halfway through the semester. At the end of the activity, each student submitted the original research paper with peer review sheets and the student researcher form attached. Each student also submitted the

Peer Evaluation.. 1341

questionnaire that evaluated the activity. Findings Meaningful Peer Feedback To address this area, the student researchers were asked to respond to three questions: 1. Was the feedback helpful? 2. Was the feedback constructive? 3. Was the feedback clear and understandable? All of the student researchers agreed that the feedback was helpful, constructive, and clear and understandable. They felt most strongly about the constructiveness of the feedback (84% strongly agreed) and least strongly about how clear and understandable the feedback was (only 58% strongly agreed). (See Table 2) In addition to the above, the peer evaluators were asked if they were able to give constructi ve feedback and whether or not they felt that their feedback should be useful in helping the student researcher revise his/her

paper. All of the peer evaluators responded positively to both questions. Although 100% of them strongly agreed that they gave constructive feedback, only 79% strongly agreed that the student researcher should be able to use their feedback to revise the research paper. (See Table 3)

Based on the responses ofthe student researchers and peer reviewers, the peer evaluation activity was meaningful and effective in this case. The student researchers found the feedback to be constructive and clear and therefore useful and the peer reviewers were confident that their feedback wasconstructive and helpful to the student researcher. Peer Evahiator Lessons The peer reviewers were asked if they understood their role as evaluators and 100% of them agreed they they did (68% strongly agreed). However, the responses ofthe student researchers who also served as peer reviewers

Table 2 Student Researcher Responses (N=I9) Statement 1. This activity was helpful in my revisions for this paper 2. The feedback from my peers was constructive

SD

D

U

A 32%

SA 68%

-

-

-

16%

84%

42%

58%

3. The feedback from my peers was clear and understandable

Table 3 Peer Evaluator Responses fN= 19) Statement 1.1 understood my role as a peer evaluator 2.1 was able to give constructive feedback 3. My feedback should be useful for revising the paper

SD

-

D

-

U -

A 32%

SA

-

21%

100% 79%

68%

342/ Journal of Instructional Psychology, Vol. 32, No. 4

were more illuminating. The comments can be categorized into the areas of "content" and "process". Statements about tbe content dealt witb the research topic and the material written to support tbe stated bypotbesis. Statements about the process dealt witb tbe evaluation of tbe activity. Example statements of "content" were as follows: • I must work barder to cover every aspect of this paper so tbat the reader will not wonder about any validity of my bypotbesis. • I need to analyze and critique more studies for my paper. • I need to define more words for tbe reader. • It's strange bow some things make perfect sense to you, tbe author, but aren't quite clear to another person reading the article. • I'm not sure I agree with some of the suggestions from my reviewers. • I learned a lot about other topics, like retention and tbe use of the Ligbtspan program. Example statements of "process" were as follows: • After I received my peer evaluations, the evaluators suggested I change the same tbings I was already considering cbanging. • It's bard to look at your own paper objectively. • I enjoyed this more than I tbougbt I would. I was able to see that I actually knew what I was doing and that I was getting my points across. • I am learning that we are all struggling together. I do need tbe critiques of others because if 2 people see that something is missing, I know it is not there. • The job of critiquing is not easy. • I was able to view tbe otber students' papers from a position of critic that increased my level of researcher adeptness.



From looking at and reviewing otber research papers, V ve learned a lot about tbe process and how things should flow.

It is clear from the comments tbat tbe students learned tbings about tbe papers as well as about the research process. Some students realized that clarity is extremely important but not always easy to acbieve. Some students also found that the review process helped them do a better job as researchers since tbey gained a better understanding of bow their papers would be reviewed. Impact ofthe Peer Evaluation Activit}' All of tbe students indicated tbat tbey intended to use tbe feedback to revise tbeir papers and tbat they felt that the activity helped them become better researcbers. Overall tbe students in tbis course felt that peer evaluations were helpful, meaningful and effective. I noticed tbat many of tbe students were very cautious of tbe comments that they made on tbeir peers' papers. Many asked me if it was all rigbt to mark on the paper and to write certain comments. I felt tbat these comments stemmed from their own fear of receiving written comments on tbeir papers. Tbey were likewise very apologetic to their peers wben clarifying their written comments. Similarly, tbe students as a wbole were still "learning the process" of conducting research and welcomed comments from someone otber than the instructor. They viewed tbe peer evaluator as someone wbo knew the process as tbey knew it. It seemed to help to have another set of eyes to view their work. The ensuing discussion between the student researcher and peer evaluator also allowed the researcher to clarify any content that was unclear or confusing to the peer reviewer. The peer evaluators felt that tbis activity was belpful in helping tbem in tbe final revisions for their papers. They felt tbat tbe activity was a good way to evaluate tbeir

Peer Evaluation .. 1343

understanding of the information that they had learned up to that point in the semester. Likewise, the students felt that it allowed them to be "facilitators" to others learning the process of conducting research. In general, this peer evaluation activity proved to be very effective as another way to evaluate student research papers. It allowed the students to make corrections and revisions that were from their peers rather than the instructor. It gave them an opportunity to defend their research design and statements that they had written. It also allowed the instructor an opportunity to evaluate their skills as evaluators. The student researchers stated that the activity would make them better researchers. Overall, the quality of papers from this class was significantly higher than papers collected from previous classes. This was particularly true of the review of literature section. The peer evaluation activity helped students realize that they has to include more substantial information in this section. The selection of information for this section was more representative ofcurrent information in the field and was organized in a more logical manner than in the papers presented before the peer evaluations were initiated. In terms of the editing process and whether it continued to damage their selfesteem, this peer evaluation process helped to remove the stigma associated with the instructor editing their papers. The peer evaluation process allowed the students to experience the editing process by their peers as well as perform the editing of their peer's paper. In some strange way, it seemed to bring meaning to the value of "editing marks". As one student commented, " I am learning that we are all struggling together". As a researcher, I too am struggling to learn better ways of helping student researchers become better in the process of "doing research".

References Barrett, J. (1986) The evaluation of teachers. (ERIC Digest No. 12). Washington, D.C: ERIC Clearinghouse on Teacher Education. (ERIC Document Reproduction Service No. ED278657) Benshoff, J. (1988). The effects of the Structured Peer Supervision Model on overall supervised counseling effectiveness ratings of advanced counselors in training. Dissertation Abstracts International. 49. no II A. Accession No: AAG8902828 Boulmetis. J. and Dutwin, P. (2000). The ABCs of evaluation. San Francisco: Jossey-Bass Boyd. R. T C.( 1989). Improving teacher evaluations. (ERIC Digest No. Ill), American Institutes for Research Washington D.C: ERIC Clearinghouse on Tests Measurement and Evaluation. (ERIC Document Reproduction Service No. ED3I5431) Darling-Hammond, L.,Wise,A.E.,& Pease. S. R. (1983). Teacher evaluations in an organizational context: A review of literature. Review of Educational Reseaivh. 53(3). 285-328. Dochy. F.J.R.C. and McDowell, L. (1997). Assessment as a tool for learning. Studies in Educational Evaluation, 23. 279-298. Gay, L. and Airasian, PW. (2002). Educational research: Competencies for analysis and applications. (6'" edition) Upper Saddle River, NJ: Prentice Hall. Keig, L. and Waggoner, M. D. ( 1993 ). Collaborative peer review. The role of faculty in improving college teaching. (ERIC Digest 071). George Washington Univ. Washington DC. School of Education and Human Development. ERIC Clearinghouse on Higher Education (ERIC Document Reproduction Service No. ED378924 ) Lisk, A. R. (2000). Using teams to enhance learning in management education Retrieved March 9, 2004, from the University of Dubuque website : htip://cobacourses.creighton.edu/ MAM/200.\/papers/lisk.doc Powell, R. R. (1992). Acquisition and use of pedagogical knowledge among career-change preservice teachers. Action in Teacher Education. 13(4). 17-24. Saavedra, R & Kwun, S. K. (1993). Peer evaluation in self managing work groups. Journal of Applied Psychology. 7S(3),450 - 462. Sapin-Piane, B. M. (1993). Mature adults becoming teachers: Sailing towards Ithaka. Unpublished doctoral dissertation. University of Massachusetts, Amherst.

344 / Journal of Instructional Psychology, Vol. 32, No. 4

Index Aduwa-Ogiegbaen, S.E.O. & Isah, M.S., Extent of faculty members' use of Internet in the University ofBenin. Nigeria. 32(4). 269-76. Ajayi, L.J., A sociocultural perspective: Language arts, vocabulary activities, and English language learners in a second grade mixed classroom. 32(3), 180-95. Astleitner, H., Principles of Effective Instruction: General standards for teachers and instructional designers. 32(1), 3-8. Augustyniak, K., Murphy, J., & Phillips, D.K., Psychological perspectives in assessing mathematics learning needs. 32(4), 277-86. Boon, R.T., Ayres, K, Forek, G., & Spencer, V.G. The effects of cognitive organizers to facilitate content-area learning for students with mild disabilities: A pilot study. 32(2), 101-117. Bums, M.K. & Dean. V.J., Effect of drill ratios on recall and on-task behavior for children with learning and attention difficulties. 32(2), 118-26 ChanLin. LJ., Development of a questionnaire for determining the factors in technology integration among teachers. 32(4). 287-92. Clump, M.K., Changes to students' learning processes following instructions on the topic. 32(4), 293-7. Corpus. J.H. & Eisbach. A.O..A live demonstration to enhance interest and understanding in child development. 32(1). 35-43. Diamantes, T., Open record laws and the tenure and promotion process. 32(3), 196-7. Dykeman. B.F. Cultural implications of crisis intervention. 32(1), 45-48. Ediger, M., Themes to emphasize in the geography curriculum. 32(2), 160-3. Egbocbuku, E.O. & Obodo, B.O. Effects of systematic desensitization (SD) therapy on the reduction of test anxiety among adolescents in Nigerian schools. 32(4), 298-304. Findlay H.F., Analysis of teachers' concerns in selected Alabama schools. 32(2), 152-9 Fore. C. & Riser, S.E., Promoting maintenance and generalization tbrough cognitive decision making training. 32(2), 127-35. Gadzella, B.M., Stacks, J., Stephens, R.C. & Masten, W.G. Watson-Glaser Critical Tbinking Appraisal, Form S for education majors, 32(1), 9-12. Geiger, J.F. & Litwiller, R.M. Spatial working memory and gender differences in science. 32(0,49-57. Germain, M.L. & Scandura, T.A., Grade inflation and student individual differences as systematic bias in faculty evaluations. 32(1), 58-67. Heffner, M. & Cohen, C , Evaluating student use of web-based material. 32(1), 74-81. Iyamu, E.O.S. & Obiunu, J.J. Impact of citizenship education on the civic consciousness of Nigerian youth. 32(4). 305-9. Kelly, W.E., An investigation of noctcaelador and vocational interests. 32(2), 164-6. Kranz, PL., Steele, R.A. & Lund, N.L., The Venture Studies Program at the University of North Florida: Innovation, Experiential courses. 32(1), 17-20. Kuhn, L., Kranz, PL., Koo, F., Cossio, G., & Lund, N., Assessment of stress in physician assistant students. 32(2), 167-77. Lane, R.J., Bishop, H.L.. & Wilson-Jones,L., Creating an effective strategic plan for tbe school district. 32(3), 197-204. Lee, D. & Kang, S., Perceived usefulness and outcomes of Intranet-Based Learning (IBL): Developing asynchronous knowledge management systems in organizational settings. 32(1), 68-73.

Journal of Instructional Psychology, Vol. 32 , No. 4 . . / 345

Machen, S.M.. Wilson, J.D.. & Notar, C.E., Parental involvement in the classroom. 32(1), 13-6. McDonald, T.W.. Caso, R., & Fugit, D., Teaching and learning operant principles in animal shelters: Perspectives from faculty, students, and shelter staff. 32(4), 310-21. Oh, D.M., Ankers, A.M., Llamas, J.M., &Tomyoy, C , Impact of pre-service student teaching experience on urban school teachers. 32(1), 82-98. Pan, W. & Tang, M., Students' perceptions on factors of statistics anxiety and instructional strategies. 32(3). 205-14 Panasuk, R.M. & Todd. J., Effectiveness of lesson planning: Factor analysis. 32(3). 21538. Parente, D.H.. Brown, R.C, & Warner, A.G., Review sessions and results: Testing the Capstone Business Class. 32(3), 233-43. Payne, E.E. & Payne, J.S., A comparison of lecture ratings by native speakers of English with EFL students at two universities. 32(3). 244-7. Petress, K.. The role of censorship in school. 32(3). 248-52. Pope, J. & Wilder, J., Now that I'm out in the field: Student teaching and valuing diversity. 32(4), 322-8. Quezada, R., From student teacher to teacher: Making the second cut (Part II). 3(1), 219. Ramirez, S.Z., Flores-Torres, L.L., Kranz, P.L. & Lund, N.L., Using Axline's Eight Principles of Play Therapy with Mexican-American children. 32(4), 329-37. Reese-Durham, N., Peer evaluation as a active learning technique. 32(4), 338-43. Schoen, A. A., Culturally sensitive counseling for Asian Americans/Pacific Islanders. 32(3), 253-8. Smith-Burton, K., Using student peer evaluations to evaluate team taught lessons. 32(2), 136-8. Snell, J. & Marsh. M., Biochemical sociology: Correlates of aggression. 32(1), 30-4. Swain, K.D., CBM with goal setting: Impacting students' understanding of reading goals. 32(3), 259-65. Swars, S.L., Examining perceptions of mathematics teaching effectiveness among elementary preservice teachers with differing levels of mathematics teaching efficacy. 32(2), 139-47. Warren, L.L. & Holloman, H.L. Jr., On-line instruction: Are the outcomes the same? 32(2), 148-51. Watts, S.J. & Markham, R.A.. Etiology of depression in children. 32(3), 266-70.