March 10th, 2011

The Facts on Higher Order Thinking


I just read a study that pretty much blew my socks off. An article highlighting the details will appear in the March issue of The Teaching Professor. I’ll give you the nutshell version here. The researchers were interested in finding out if there was empirical evidence to support the frequent criticism that introductory courses are fact filled with little content that challenges higher order thinking. Beyond anecdotal evidence, this research team didn’t find much empirical documentation so, being biologists, they decided to look at introductory-level biology courses.

They collected syllabi, tests and quizzes from 50 faculty who taught introductory courses (about 50% general biology and the rest a range of courses, including environmental science and cell and molecular biology), and they taught at different types of institutions. The researchers looked at the goals listed in the syllabus and rated them according to the levels on the Bloom taxonomy. They did the same thing for exam and quiz questions—9,713 of them. Would you believe 93% of those test questions were rated at levels 1 or 2, the knowledge and comprehension levels of Bloom’s taxonomy. Less then 1% were above level 4. As for the goals on the syllabus, 69% of those were also at the same two lowest levels. For both the test questions and goals, class size and institutional type didn’t make any difference.

Is this a fair way to analyze what’s being emphasized in a course? The researchers answer this way. “These data provide evidence of what faculty consider important in courses. Goals stated in syllabi reflect faculty priorities about what they expect students to know and be able to do; assessments reflect how faculty evaluate students’ achievement of those learning goals.” (p. 436)

Do I hear you thinking, these data are about biology courses and not all disciplines are the same? True. Has data been collected in your discipline verifying something different? In the absence of data, we have opinions and anecdotal evidence. I would rather talk about the valuable questions raised when a discipline does an analysis like this.

Are beginning courses the best time to teach facts? Must students know the facts before they can think at higher levels? Asked a bit differently, is knowing the facts all that’s needed to think at higher levels? Must students practice making connections, integrating facts and applying information, or can they do that automatically is they’ve got the facts?

What we’re really dealing with here is a long-held assumption that knowledge of a discipline begins with the facts, those bedrock basics. I certainly wouldn’t want to debate the necessity of knowing the facts of a field. The question is how and when that factual knowledge base ought to be developed. What best challenges the veracity of a fact-filled introductory course is the research about how few of those facts students retain after the course is over. Some of us have observed facts vanish the day after the exam.

We can read research like this and think about it somewhat theoretically. We can ask questions that lead us to re-examine the role of content in learning—a valuable exploration indeed. But I also think a study like this is cause for personal reflection and analysis. What about the goals on your syllabi? What about the questions on your exams?

It’s good to remember that we’re no different from other people. Honesty and objectivity about what we do does not always come easily. I am thinking about one of my favorite studies—an analysis of grade inflation where faculty responded that at their institution and in their discipline it was a big problem, but it wasn’t a problem in their classroom. An analysis of their grades revealed a different truth. Trusted colleagues can be of great help when it comes to getting a clear-eyed view of our instructional practices.

Reference: Momsen, J. L., Long, T. L., Wyse, S. A. and Ebert-May, D. (2010). Just the facts? Introductory undergraduate biology course focus on low-level cognitive skills. Cell Biology Education—Life Sciences Education, 9 (Winter), 435-440.