designing test questions
The goal of any well-constructed test is to test students’ expertise on a topic and not their test-taking skills. We need to eliminate as many flaws in our questions as we can to “provide a level playing field for testwise and not-so-testwise students. The probability of answering a question correctly should relate to an examinee’s expertise on the topic and should not relate to their expertise on test-taking strategies.” (NMBE, 2001, p 19)
I remember with horror and embarrassment the first multiple-choice exam I wrote. I didn’t think the students were taking my course all that seriously, so I decided to use the first exam to show just how substantive the content really was. I wrote long, complicated stems and followed them with multiple answer options and various combinations of them. And it worked. Students did poorly on the exam. I was pleased until I returned the test on what turned out to be one of the longest class periods of my teaching career. I desperately needed the advice that follows here.
As Ron Berk (known for his pithy humor) observes, the multiple-choice question “holds world records in the categories of most popular, most unpopular, most used, most misused, most loved and most hated.” According to one source I read, multiple-choice questions were first used around the time of World War I to measure the abilities of new Army recruits. As class sizes have grown and the demands on teacher time expanded, they have become the favorite testing tool in higher education.