It’s good to regularly review the advantages and disadvantages of the most commonly used test questions and the test banks that now frequently provide them.
The relatively new Scholarship of Teaching and Learning in Psychology journal has a great feature called a “Teacher-Ready Research Review.” The examples I’ve read so far are well organized, clearly written, full of practical implications, and well referenced. This one on multiple-choice tests (mostly the questions on those tests) is no exception. Given our strong reliance on this test type, a regular review of common practices in light of research is warranted.
They’re the kind of questions that promote thinking and result in sophisticated intellectual development. They’re the kind of questions teachers aspire to ask students, but, according to research, these types of questions aren’t the typical ones found on most course exams. Part of the disconnect between these aspirations and the actualities results from the difficulty of writing questions that test higher-order thinking skills.
The goal of any well-constructed test is to test students’ expertise on a topic and not their test-taking skills. We need to eliminate as many flaws in our questions as we can to “provide a level playing field for testwise and not-so-testwise students. The probability of answering a question correctly should relate to an examinee’s expertise on the topic and should not relate to their expertise on test-taking strategies.” (NMBE, 2001, p 19)
