April 29th, 2010

Ratings: Working on the Cynicism


If you’re on a semester calendar, this academic year is winding down. As courses come to a close, it’s time for those end-of-course ratings which many of us administer with some cynicism.

At lot of institutions now students do the rating forms online, and as a consequence the number of students who complete them has plummeted. Why should they when they see that their feedback has little impact, most especially on those teachers who need to improve? (You’re detecting more cynicism?) I did discover an effective solution to the problem of low response rates, though. The authors of a research article noted in passing that at their institution students who completed the online course ratings instrument got earlier access to their grades than those who did not. Worried that that might bias the sample with those grade-hungry students being the only ones who completed the course evaluation? That wasn’t the case at this place: the overall student response was almost 80 percent—a sample size large enough to be representative.

I have written a lengthy article for the newsletter on the research undertaken in this study. It offers the first viable model I’ve seen for dealing more systematically and objectively with the student comments collected on most of these end-of-course forms. They start out by asking students good open-ended questions: “What are the teacher’s strengths?” and “What improvements would you suggest to the teacher?” I wouldn’t give those questions an A—the best questions ask students to connect instructional approaches, assignments, and other aspects of the course to learning experiences: “How did the instructional approaches used in this class affect your efforts to learn?” But the questions used in the research were far superior to those questions that inquire about what students “liked” most and least about a course and/or its instructor.

One of the best ways to overcome the cyncism so many of us feel about end-of-course summative evaluations is to see them as part of the picture—one window offering a view of the teaching. To make the picture more complete and accurate, teachers need to cut out many windows seeking feedback from students in different ways, at different times, and on different topics. It also helps to remember that these ratings are assessments of experiences in a course—they do not measure our worth as human beings.

Look for the article summarizing the research mentioned here in the June-July issue of the newsletter, or check out the article for yourself.

Pan, D., Tan, G. S. H., Ragupathi, K., Booluck, K., Roop, R. and Ip, Y. P. ( 2009). Profiling teacher/teaching using descriptors derived from qualitative feedback: Formative and summative applications. Research in Higher Education, 50 (1), 73-100.