Many faculty incorporate a peer-assessment component in team projects. Because faculty aren’t present when the groups interact and therefore don’t know who’s doing what in the group, they let students provide feedback on the contributions of their group-mates. In addition to giving the teacher accurate information on which to base individual grades, the process gives students the opportunity to learn the value of constructive peer feedback. It’s a skill applicable in many professional contexts.
Most faculty have discovered that the quality of peer feedback improves if students use a form that articulates assessment criteria. Otherwise, given a form that asks them to rate or describe the contributions of other members, students tend to avoid giving negative feedback and to fall back on the “everybody contributed equally” mantra.
A group of faculty (mostly in engineering) looked at the inter-rater reliability of three short peer-evaluation forms. Inter-rater reliability is a statistical measure of the extent of agreement among evaluators. It’s an important feature of good assessment instruments. One of the forms used was a single-item instrument without any behavioral anchors or specific assessment criteria, similar to what’s described in the previous paragraph. The second form used a five-point rating scale and asked students to assess team members across 10 categories that included various behaviors, e.g., attended group meetings regularly, contributed to discussions, listened effectively, performed significant tasks, and completed tasks on time. The final form included these kinds of behavioral anchors in its instructions and elaborated descriptions of the rating words (excellent, for example, was defined as “consistently went above and beyond, tutored teammates, carried more than his or her fair share of the load”). However, on this form group members gave peers a single rating assessment. Researchers found that both behaviorally anchored forms had about the same high inter-rater reliability when they were used by four raters in the same group.
The value, of course, is the economy of the shorter form. It considerably expedites the grading process, which benefits instructors who may have large classes and homework and other assignments to grade. Researchers also hypothesize that students will complete shorter forms more conscientiously. However, they do recommend using the longer form to accomplish formative goals. They have their students complete it at the end of a first project so that group members can use the feedback to identify areas for improvement. If the feedback indicates that a group has some members who are “hitchhiking” (as in getting a free ride from the group) or “overachieving” (as in dominating and overdirecting the group effort), the instructor meets with those groups to explore better ways to distribute the workload and leadership within the group.
[report_ofie=2440]
All three of the forms tested in this analysis are included in this article.
Reference: Ohland, M. W., Layton, R. A., Loughry, M. L., and Yuhasz, A. G. (2005). Effects of behavioral anchors on peer evaluation reliability. Journal of Engineering Education, 94(3), July, 319–325.