April 9, 2008
The Disconnect Between Faculty Beliefs and Research on Student Ratings
I’m knee deep, some days it feels more like waist deep, in the research on student ratings. I’m still on that chapter on ratings for my book. What a sea of information!
Besides being in deep, I’m also still stuck on the disconnect between what faculty believe and what the research documents. And these aren’t disconnects over trivial points. They are the big ticket items. Case in point: Is the way to win at the ratings game by teaching an easy course? One where the content isn’t all that difficult, where the workload isn’t all that heavy, and where the good grades are pretty easily acquired?
That’s what most faculty think. A 1987 study (Marsh) found that 72 percent of faculty at a research university thought that course difficult biased ratings, 68 percent said the same about grading leniency, and 60 percent believed that courses requiring more work received lower ratings. In a 2002 study (Sojks, Gupta and Deeter-Schmelz) 53 percent of faculty agreed or strongly agreed that students give better rating to instructors that teach less demanding courses.
I suppose that’s progress, but research repudiating all these beliefs is so compelling. For example, look at recent studies by Centra (2003), and Marsh and Roche (2000), three widely respected and prolific researchers in this area. These studies are giant. Centra considered data from 50,000 different courses. His overall conclusion: “[T]eachers will not likely improve their evaluations from students by giving higher grades and less course work.” (p. 526) Marsh and Roche actually found the courses that demand the least amounts of work receive lower ratings.
Research studies like Centra, and Marsh and Roche are not easy reading if you know little or nothing about social science research—they are long and excruciatingly detailed even if you do. But they offer hard evidence. Isn’t it amazing that in a culture that so prizes evidence, proof, and documentation, myths about important aspects of teaching continue to be promulgated?
Marsh, H. W. “Students’ Evaluations of University Teaching Research Findings, Methodological Issues, and Directions for Future Research.” International Journal of Educational Research, 1987, 11, 253-388.
Sojka, J., Gupta, A. K., and Deeter-Schmelz, D. R. “Student and Faculty Perceptions of Student Evaluations of Teaching: A Study of Similarities and Differences.” College Teaching, 2002, 50 (2), 44-49.
Centra, J. “Will Teachers Receive Higher Student Evaluations by Giving Higher Grades and Less Course Work?” Research in Higher Education, 2003, 44 (5), 495-519.
Marsh, H. W., and Roche, L. A. “Effects of Grading Lenience and Low Workload on Students’ Evaluations of Teaching: Popular Myth, Bias, Validity, or Innocent Bystanders?” Journal of Educational Psychology, 2000, 92 (1), 202-228.