January 20th, 2009

Student Rating Reminders


Student evaluation results from last semester usually get distributed about this time at a lot of places. I thought a few reminders might be in order. This particular set is brought to you courtesy of the IDEA Center (Individual Development and Assessment Center) at Kansas State University. This center is one of the largest providers of a national student rating system. They have been in business for years and are highly respected by both ratings researchers and practitioners.

  • Even though the center is in the business of selling a rating system, they are adamant that student ratings should comprise somewhere between 30 percent and 50 percent of an overall assessment of teaching. Why? Because students aren’t qualified to judge some aspects of teaching, like whether the instructor has selected an appropriate text for the course, and because, despite the reliability and validity of a well-designed form, they simply don’t tell the whole story.
  • Small differences in rating scores (like the difference between a 4.0 and a 4.1) are meaningless. William Pallette who runs the center says this is like using a razor to cut a log. He recommends dividing rating results into three or five large categories: “outstanding,” “exceeds expectations,” “meets expectations,” “need improvement but is making progress,” and “fails to meet expectations.”
  • In order for evaluations to fairly represent teaching, they must include results from six to eight courses, eight to 12, if the courses are small. Overall assessments of teaching competence based on smaller data sets may not be representative.
  • In order for a set of ratings to be representative of what happened in a particular course, response rates need to be above 65 percent.

Reminders like these are important when you look at your rating results. It is also important that faculty committed to teaching and interested in advancing instructional causes be knowledgeable about ratings. I don’t think I have to remind you that not all faculty or administrators are.

Reference: Pallett, W. Uses and abuses of student ratings. In P. Seldin and Associates, Evaluating Faculty Performance: A Practical Guide to Assessing Teaching, Research and Service. Bolton, Mass: Anker, 2006. (Note: Anker is now a part of Jossey-Bass. This may be ordered from www.josseybass.com)