April 9th, 2008

The Disconnect Between Faculty Beliefs and Research on Student Ratings


I’m knee deep, some days it feels more like waist deep, in the research on student ratings. I’m still on that chapter on ratings for my book. What a sea of information!

Besides being in deep, I’m also still stuck on the disconnect between what faculty believe and what the research documents. And these aren’t disconnects over trivial points. They are the big ticket items. Case in point: Is the way to win at the ratings game by teaching an easy course? One where the content isn’t all that difficult, where the workload isn’t all that heavy, and where the good grades are pretty easily acquired?

That’s what most faculty think. A 1987 study (Marsh) found that 72 percent of faculty at a research university thought that course difficult biased ratings, 68 percent said the same about grading leniency, and 60 percent believed that courses requiring more work received lower ratings. In a 2002 study (Sojks, Gupta and Deeter-Schmelz) 53 percent of faculty agreed or strongly agreed that students give better rating to instructors that teach less demanding courses.

I suppose that’s progress, but research repudiating all these beliefs is so compelling. For example, look at recent studies by Centra (2003), and Marsh and Roche (2000), three widely respected and prolific researchers in this area. These studies are giant. Centra considered data from 50,000 different courses. His overall conclusion: “[T]eachers will not likely improve their evaluations from students by giving higher grades and less course work.” (p. 526) Marsh and Roche actually found the courses that demand the least amounts of work receive lower ratings.

Research studies like Centra, and Marsh and Roche are not easy reading if you know little or nothing about social science research—they are long and excruciatingly detailed even if you do. But they offer hard evidence. Isn’t it amazing that in a culture that so prizes evidence, proof, and documentation, myths about important aspects of teaching continue to be promulgated?

Marsh, H. W. “Students’ Evaluations of University Teaching Research Findings, Methodological Issues, and Directions for Future Research.” International Journal of Educational Research, 1987, 11, 253-388.

Sojka, J., Gupta, A. K., and Deeter-Schmelz, D. R. “Student and Faculty Perceptions of Student Evaluations of Teaching: A Study of Similarities and Differences.” College Teaching, 2002, 50 (2), 44-49.

Centra, J. “Will Teachers Receive Higher Student Evaluations by Giving Higher Grades and Less Course Work?” Research in Higher Education, 2003, 44 (5), 495-519.

Marsh, H. W., and Roche, L. A. “Effects of Grading Lenience and Low Workload on Students’ Evaluations of Teaching: Popular Myth, Bias, Validity, or Innocent Bystanders?” Journal of Educational Psychology, 2000, 92 (1), 202-228.

–Maryellen Weimer

  • Derek

    So I'll buy that easier courses don't generate higher student ratings, but I'm not a typical faculty member. I work at a teaching center, so what can I say to a faculty member that will convince them of this? I don't think the hard data is as convincing to faculty as some kind of reasoning that can counteract the intuition they have about ratings. What attitudes or beliefs about students are required for someone to say, "Yeah, those results make sense to me"?

  • Jennifer Imazeki

    I wonder if part of the reason faculty believe that ratings are higher for easy classes is because students will say that they do want easy classes and often recommend easy classes to their friends – as an undergraduate advisor, I have had students tell me outright "Oh, my roommate said I should take X's class because it's really easy." But the key is that students don't want easy classes because they don't want to learn – they want easy classes because they only have so many hours in a day and they often are looking for 'easy' classes in order to offset harder classes that they already plan to take. I think the other problem faculty have with believing the data is when it just doesn't match their own experience. That is, I'm assuming that most faculty who believe that easier classes get better ratings are faculty who, themselves, are not getting ratings as high as they would like. It's far easier to believe that your ratings are low because your class is hard than to believe that it's something about the way you teach. If that is the case, I'm not sure there really is much you can do to convince someone that their 'intuition' is wrong…