March 10th, 2011

The Facts on Higher Order Thinking


I just read a study that pretty much blew my socks off. An article highlighting the details will appear in the March issue of The Teaching Professor. I’ll give you the nutshell version here. The researchers were interested in finding out if there was empirical evidence to support the frequent criticism that introductory courses are fact filled with little content that challenges higher order thinking. Beyond anecdotal evidence, this research team didn’t find much empirical documentation so, being biologists, they decided to look at introductory-level biology courses.

They collected syllabi, tests and quizzes from 50 faculty who taught introductory courses (about 50% general biology and the rest a range of courses, including environmental science and cell and molecular biology), and they taught at different types of institutions. The researchers looked at the goals listed in the syllabus and rated them according to the levels on the Bloom taxonomy. They did the same thing for exam and quiz questions—9,713 of them. Would you believe 93% of those test questions were rated at levels 1 or 2, the knowledge and comprehension levels of Bloom’s taxonomy. Less then 1% were above level 4. As for the goals on the syllabus, 69% of those were also at the same two lowest levels. For both the test questions and goals, class size and institutional type didn’t make any difference.

Is this a fair way to analyze what’s being emphasized in a course? The researchers answer this way. “These data provide evidence of what faculty consider important in courses. Goals stated in syllabi reflect faculty priorities about what they expect students to know and be able to do; assessments reflect how faculty evaluate students’ achievement of those learning goals.” (p. 436)

Do I hear you thinking, these data are about biology courses and not all disciplines are the same? True. Has data been collected in your discipline verifying something different? In the absence of data, we have opinions and anecdotal evidence. I would rather talk about the valuable questions raised when a discipline does an analysis like this.

Are beginning courses the best time to teach facts? Must students know the facts before they can think at higher levels? Asked a bit differently, is knowing the facts all that’s needed to think at higher levels? Must students practice making connections, integrating facts and applying information, or can they do that automatically is they’ve got the facts?

What we’re really dealing with here is a long-held assumption that knowledge of a discipline begins with the facts, those bedrock basics. I certainly wouldn’t want to debate the necessity of knowing the facts of a field. The question is how and when that factual knowledge base ought to be developed. What best challenges the veracity of a fact-filled introductory course is the research about how few of those facts students retain after the course is over. Some of us have observed facts vanish the day after the exam.

We can read research like this and think about it somewhat theoretically. We can ask questions that lead us to re-examine the role of content in learning—a valuable exploration indeed. But I also think a study like this is cause for personal reflection and analysis. What about the goals on your syllabi? What about the questions on your exams?

It’s good to remember that we’re no different from other people. Honesty and objectivity about what we do does not always come easily. I am thinking about one of my favorite studies—an analysis of grade inflation where faculty responded that at their institution and in their discipline it was a big problem, but it wasn’t a problem in their classroom. An analysis of their grades revealed a different truth. Trusted colleagues can be of great help when it comes to getting a clear-eyed view of our instructional practices.

Reference: Momsen, J. L., Long, T. L., Wyse, S. A. and Ebert-May, D. (2010). Just the facts? Introductory undergraduate biology course focus on low-level cognitive skills. Cell Biology Education—Life Sciences Education, 9 (Winter), 435-440.

  • It's unfortunate that this discussion is in fact restricted to natural science classes, which will be somewhat different in these matters from the humanities or core classes I and my colleagues teach — or even from the social science classes of my other colleagues. Particularly in natural science classes, students report having a lot of difficulties even getting through the Bloom level 1 and 2 activities and to the goals — retaining the core, baseline, fundamental knowledge, distinctions, concepts, definitions — needed to go any further. Actually, they do have similar problems in my own classes — but they have even greater problems in the natural science classes, not least since there's more concepts to learn (I'll not say master, since that rarely takes place) in order to be able to do anything interesting in the class.

    I'm guessing that if you really intend to "question is how and when that factual knowledge base ought to be developed" in intro-level classes, you've not recently taught class after class of the students coming into our middle- and lower-tier colleges and universities, who not only lack much of the badly needed factual knowledge they ought to have been taught in middle and high school (so that we have to do an inordinate amount of purely remedial work), but already have well-developed bad intellectual habits keeping them from even grasping that they do need to memorize, practice, drill, self-assess before going on to more engaging higher-order activities.

    Many of the students I see semester after semester in my Critical Thinking class flunk precisely because they are unable — or rather, don't make the series of consistent, connected choices needed — to carry out as simple an operation as identifying a conclusion in an argument — after we have discussed study skills, actually done some (engaging, higher order, context-providing) practical reasoning about what it takes to do well in the class, to develop the skills employers say they are looking for (and all of the students in the kind of place I teach when asked are very clear about why they are there — they believe college is the ticket to a better life via a good job) — and actually after we've done a number of examples, reinforced over class sections – and even after giving them a study guide that tells them precisely what operations they have to be able to do on the test.

    We've experimented and reconfigured that particular course in many different ways over the last several years, using a host of different pedagogical strategies aimed at situating lower-order activities within more engaging higher-order activities and contexts — and what usually happens is that two things get reconfirmed. First, that if you don't get down the basic concepts, distinctions, operations that comprise the fundamental knowledge for the class, you can't effectively do anything higher-order either — though you can certainly be misled into feeling like you can. Second, that most of our students, even when given ample time, all sorts of support, models for good and poor performance, etc. aren't willing to put in the time, work, and focused attention needed to develop proficiency at the lower level.

    I'd suggest that for this present generation of students — setting aside the ones who did get an adequate K-12 education, who have some decent level of developed basic skills needed in order to study material, retain information, and even transfer knowledge and skills from one context to another (fairly rare in the students I see), we have to concentrate on the lower-order. I've actually come think that in the majority of cases, we even do a disservice to students by short-changing lower-order activities in favor of focusing on higher-order ones, when we should be focusing much more on how to better teach the lower-order skills and knowledge — in ways that can be demonstrated.

    Would I like for myself and my colleagues to be able to focus more on higher-order learning and to bring our students along into that, collaboratively, engagingly? I would love it. Who wouldn't? But that's not what most of our students are ready for, and it is not what they need at the present time. If I were to find a position where I taught better prepared students, I'd already be doing what you advocate here. But, one has to teach to the students one actually has, to get them a bit further along in the time that one has them. I can just imagine the situation my colleagues in natural sciences are in, and I would in fact support them — at least with our students — in ignoring your advice and "sticking too the facts"

  • Larry Spence

    The big question is how factual knowledge should to be developed. Bloom’s taxonomy is no help. Basic concepts are abstract, complex and cannot be learned very well, if at all, unless they are imbedded in reasoning. The lower/higher distinction obscures the problem and produces the false solution: memorize the facts then use them to think. Learning involves thinking. I submit that students can’t learn concepts, definitions, distinctions, core knowledge, etc. unless they practice thinking about them, using them, and struggling through the processes of understanding them. Students must think with the facts in order to remember them usefully.

    I taught a critical reasoning course in political science for honors students for more than fifteen years. The course was born from failure. When the university initiated an honors program that required a senior thesis results were horrific. Our best students could not carry out, even under faculty guidance, competent research. We tried to remedy that with a required course. From the beginning I faced students with the best SAT scores and the highest GPA’s. But they either couldn’t relate the facts they recited to anything like a research question or were innocent of anything but bloated opinions about policies and institutions.

    I could hardly blame these best and brightest. Colleagues cared more about student competence than what I covered or if students rated my teaching highly. Those conditions were ideal. But I had to ask questions about how to develop basic knowledge and get some answers. I found that concepts are difficult units of knowledge. If students just memorize names and definitions they don’t know enough to apply or evaluate them. They may talk like social scientists, much the way computers can be programmed to talk like therapists, but they cannot investigate or understand the political world. I learned to incorporate the basics in collaborative activities and provide lots of directed practice and coaching. In short, the course began and ended with students doing interesting things.

    This approach is confirmed by the work of many natural and social scientists who work with concept inventories to improve their courses. As developed by Eric Mazur at Harvard, this method requires students to resolve misunderstandings and collaborate to master new ideas and skills. Data obtained in introductory classes in various academic settings and disciplines – including the natural sciences, statistics, and social sciences – show that peer learning approaches triple learning gains over a semester compared to traditional courses. Such courses are demanding of both faculty and students, but the evidence suggests they work. So why don’t we skip Bloom and use them?

  • Renee

    I teach in the natural sciences, and in addition to facing a lack of student interest, I also find that I am required to cover so much content that the only thing I have time to do is teach the facts. Honestly, I have lapsed on this content, in order to give students time to work with the concepts –I cover fewer chapters than I am supposed to. It makes for a better teaching experience, and for those students who can muster some interest in their world, a better learning experience.

    Once again, however, as Gregory Sadler commented, what and how we teach is very dependent on the the preparation of our students. Most public schools spew out students who have been taught in ways that emphasize that learning is boring and difficult, and these students have been disengaged for years. Even the students who want to understand how a process works, rather than just recite the facts, have trouble doing this because their previous education has left them unprepared for reasoning at the college level.

  • Pingback: Teo-Education.Com » Blog Archive » The Facts on Higher Order Thinking()

  • Daniel

    In 1st grade, I was placed in a gifted program titled called "Project Potential," where instead of teaching us facts, we were exposed to logic games, brain teasers, and occasionally a research project. I'm 25 years old now, and I can still remember some of the Eureka! moments I had in those classes, and there's no doubt in my mind that a great deal of the successes I've had so far have been due to their decision to develop my mind rather than fill it with information.

    After going through that program at such a young age, I learned to look for connections between facts, figures, and estimations, and I think that prepared me for better fact retention later in life. Higher order thinking should be a foundation of learning for students–not unlike a company's decision to invest in the right tools before beginning a project. In the end, it pays off in spades.

  • Kenneth Keeling

    I teach a course – psychopathology – at an online university. The course is part of a Master's program in Counselling (yes in Canada it is spelled with 'll') Psychology. Many of the students come from non-psychology undergraduate backgrounds. I would like to say that I have not found a great difference between those without a psychology background and those with – both groups demonstrate little in the ability to apply higher order thinking skills to the course material. At the Master's level on would expect that students are able to function at higher than Bloom's levels 1 and 2. Alas this is not the case. The majority of students have little capacity to analyze and synthesize the course material. – they struggle just with basic understanding of the assigned reading.

    I have ome to the conclusion that for most students, to expect functioning above Level 2 is close to futile – the 10 or 15 % who do are a joy. I must admit that the 20% or so who can't even master material at Levels 1 and 2 are a concern. These students will in all likelihood graduate and get jobs in the counselling field. They will be dealing with clients without much more than a rudimentary understanding of psychopathology. One can only hope that that the required practicum experience will instil sensitivity to clinical issues.

    As result of my experience, I am beginning to consider a U-turn. To this point my focus for Discussion Forum topics and for the Major Paper have been designed to induce higher order thinking – it isn't working for the majority of students. Most cannot meet the challenge.

    Recently I ran across a book that may be of interest to those who have been posting here. While aimed at the secondary school level, there is much to ponder for those of use who teach at the undergraduate and even the graduate level: Daniel T. Willingham – Why Students Don't Like School. Willingham is a cognitive scientist and in this pithy book provides material that supports an emphasis on Level 1 and 2 learning.

  • Pingback: Final Exams May Not Really Measure Learning - Online College Courses()