Blog

Feature Image- Designing Effective Multiple Choice Questions

Assessments

An assessment is an exercise which measures the academic performance of a learner in absolute and relative terms. It gives an idea about the extent of subject knowledge acquired by a learner. An assessment comprises questions in different formats. Multiple Choice Questions (MCQs) are one of the most widely adopted formats for developing assessments. An assessment, in part or full, may comprise MCQs. This article examines the origins of the first MCQs, a comparison of MCQs with essay type questions, and what entails designing effective multiple-choice questions.

 

The First MCQs

Text diagram

Multiple Choice Questions were not an integral element of the academic assessment systems from the beginning of the formal education system. An educator, however, realized that a student’s performance in a test or class was a matter of subjective evaluation by different teachers. The educator could make one more important observation- that the assessment process was time taking for teachers. The first ever MCQs were instrumental in getting around and over these two hurdles.  Some additional benefits of the MCQ based tests were obvious. The tests could be conducted for a very large number of examinees and the process of evaluation and assessment still did not take much time. In addition, the answers could also be graded by someone who was not necessarily a teacher. The students’ performance was no more a matter of subjective evaluation by different teachers. The MCQs thus, standardized the tests.

Readers may visit the webpages: Multiple Choice and Testing Machines: A History at HackEducation and Where Did Standardized Testing Come From Anyway? at hastac for an in-detail reading about MCQs developed.

 

MCQs vs Essay Type Questions

Text Diagram

The new MCQ based tests offered an edge over the traditional essay type tests and hence widely accepted into the education system. However, there was always a counterview. It was believed that MCQs had a lower educational value, and that answering essay type tests required examinees to not only possess but also demonstrate their depth of knowledge. Essay questions also made it incumbent upon the examinees to develop creative thinking and effective writing skills. MCQs seemingly, helped students score, merely on their ability to memorize, and not help a student develop logical and analytical thinking.

However, the public education system was expanding, thereby pushing the need to have a measure that could help meet the assessment requirements for an increasing number of students. The new education system also required a versatile mechanism to not only evaluate an individual’s progress in absolute terms, but also compare with that of other students across schools and geographies. The MCQs were thus the need of the hour.

The webpages: Where Did Standardized Testing Come From Anyway? at hastac; and Improving Your Test Questions at CITL, University of Illinois offer more information on how MCQs compare with essay type questions.

 

The Need to Evolve

The true utility of an assessment not only lies in grading examinees, but also in helping them identify their weaknesses. An assessment also provides teachers an idea about the weaknesses of their students. These valuable insights obtained from an assessment help teachers develop effective teaching strategies for specific topics. Traditional MCQ based assessments offered a faster, standardized, scalable alternative for grading students’ performance. However, the possibility of employing traditional MCQ based assessments as a tool to identify the students’ level or extent of learning was yet to be explored.

Text Diagram
Benefits of Assessments

The Structure of an MCQ

In general, the structure of a multiple-choice statement comprises a problem statement (called the stem) and four/five options listed under it. Usually, only one of the options from amongst the given list of four/five options is correct. It is the key. The rest of the incorrect options from the list are distractors.

 

Text Diagram
Structure of an MCQ

While attempting an MCQ, an examinee who has not studied well is likely to choose one of the distractors (incorrect options) as the answer. The distractors seem to be the right answer, but they are not. They are, in fact, erroneously drawn from the correct answer (the key) and hence seem to be the right answer. The choice of a particular distractor depends on the level of understanding and learning of the examinee. A well-read and prepared examinee chooses the key as answer. An examinee who has not has learnt the topic pertaining to a question properly, chooses one of distractors as the answer.

Readers may refer to the webpages:  Writing Good Multiple Choice Test Questions at the Vanderbilt University and Designing Multiple-Choice Questions at University of Waterloo to get an idea about the structure of an MCQ.

 

The Role of Distractors

An MCQ may have three/four distractors, each of which corresponds to a different level of knowledge/understanding, pertaining to a topic. The key corresponds to the complete level of knowledge/understanding of a topic. Thus, a particular distractor, when chosen as an answer, would reflect the level of knowledge/understanding of an examinee/student, pertaining to the topic of the question. The distractors can thus play a very important role in helping an instructor understand the level of knowledge/understanding student(s) have developed across one or multiple topics. A teacher employs this feedback in developing effective teaching/instruction strategy.

A detailed taxonomy of rationale (with examples) is available in the document: The Distractor Rationale Taxonomy: Enhancing Multiple-Choice Items in Reading and Mathematics at Pearson Assessments.

 

Guidelines for Designing Stem and Distractors

Text Diagram

It is important that an MCQ assesses the knowledge level of the learner effectively. This calls for a thoughtful approach while designing it. A poorly designed MCQ might give away a hint for the answer, from the very manner in which it is structured.  A well-structured MCQ would compel the learner to employ the necessary knowledge and skill to find the answer. A careful balance between different aspects of the question, such as language quality and presentation style, is thus necessary. 

An MCQ structure comprises the stem and alternatives. The stem defines the problem statement. It should be clear,precise, and sans any vagueness. Verbose and irrelevant problem statements require more time to read and understand. Such questions also cause unnecessary confusion to learners. These criteria assume more relevance for time-bound tests.

The alternatives in an MCQ comprise of the key and distractors. The choice of distractors for an MCQ has to be made thoughtfully. To begin with, the distractors should correspond to different levels of knowledge/understanding of a particular topic. They all may seem to be the correct answer. Additionally, all the distractors should be clear, have the same language as the stem, and be of the same length. Distractors should not overlap in content or carry a hint in their language, structure, or otherwise.  Due emphasis on the criteria which have been discussed while designing an MCQ is thus necessary.

The reader may refer to the webpages: Writing Good Multiple-Choice Test Questions at the Vanderbilt University and Designing Multiple-Choice Questions at the University of Waterloo websites to get more idea about designing effective multiple-choice questions.

 

Assessments are not only instrumental in assessing knowledge, but also in developing it further. Assessments can serve as a vital tool to help identify the areas in which a learner needs to make efforts to improve his overall knowledge level. This article is part of a series of articles on Assessments.

 

Additional References

  1. Advantages and Disadvantages of Different Types of Test Questions at Faculty Focus.
  2. The Role of Assessment by Fisseha Mikre.
  3. Writing Good Multiple-Choice Exams by Dawn M. Zimmaro, Ph.D., The University of Texas at Austin.
  4. The Test by Anya Kamenetz.

 

For further reading related to pedagogy and education standards, please visit the blog at Evelyn learning.

Create.Engage.Inspire

Leave a Comment

Your email address will not be published.