Translate this page into:
Designing effective scenario-based multiple-choice questions for health professionals
Correspondence to Archana Chauhan; archanachauhan46036@gmail.com
[To cite: Chauhan A, Mahajan A, Begum J, Kapoor A, Badyal D, Kulkarni S. Designing effective scenario-based multiple-choice questions for health professionals: Tips and techniques. Natl Med J India 2024;37: 335–9. DOI: 10.25259/NMJI_1004_2023]
INTRODUCTION
In recent years, India’s medical education system has experienced major transformations, primarily characterized by a transition towards competency-based learning and the incorporation of innovative assessment techniques.1 Scenario-based multiple-choice questions (SBMCQs) are becoming increasingly popular because they provide a comprehensive assessment of students’ understanding of concepts and real-world situations.2,3 With the use of SBMCQs, educators can assess application of knowledge, critical thinking, and problem-solving abilities of the students.4,5 Such an assessment emphasises practical application and fosters a deeper understanding of the subject.
Considering the potential of SBMCQs to assess students’ higher order of thinking, a competency-based curriculum for undergraduate (UG) medical students has up to 20% of the evaluation using SBMCQs.6 Many universities in India are now using SBMCQ assessments to assess the ability of students to apply theoretical knowledge to practical settings.7 A recent analysis of the national eligibility cum entrance test for postgraduation (NEET-PG) 2023 revealed that about 60% of questions were clinical scenario-based with only 10%–15% single-line questions. Further, it had over 35 picture-based questions.8
Hence, it is imperative to prepare students for this high-stakes examination by integrating SBMCQs into both formative and summative assessments throughout their UG course. Students would become more familiar with the format and gain valuable practice, increasing their chances of success in the examination. For this purpose, the faculty should be able to prepare clinical scenario-based questions addressing a higher level of knowledge in Bloom’s taxonomy. The faculty need to be able to surpass fact recall, and encourage students to analyse, evaluate, and synthesize information.9,10
However, there is a gap in training medical educators to design SBMCQs that target higher-order cognitive abilities.11,12 Our aim is to address the proficiency gap among faculty in creating SBMCQs effectively. A group of 32 faculty members from various medical colleges in India developed SBMCQs. We reviewed and validated the questions and present them here. We highlight common mistakes to avoid during this process.
KEY CONSIDERATIONS
Designing effective SBMCQs involves considering the course or competency objectives. Questions should revolve around important concepts, and common or serious clinical problems encountered in practice.2,13 Avoid trivial or rare topics that may interest clinicians but are not feasible for this type of assessment. The salient features to be considered are:
Develop a Blueprint
Match the complexity of the case to the learner’s level
Align questions with the intended learning objectives
Focus questions on common or potentially catastrophic problems
Create scenarios that mirror real-world situations
Encourage higher order thinking
Provide precise references for the question
Determine the allocation of marks or scoring system
Review, pilot test
Structure of scenario-based multiple-choice question (SBMCQ)
The SBMCQ consists of four components14,15
Stem: Presents the scenario or problem with context.
Lead-in: Intimates students on what is asked or expected.
Key: Represents the best answer or response.
Distractors: Plausible choices that distract from key.
SBMCQs help students relate their thinking to real-world situations, and promote critical thinking in their answers. However, adding a scenario without proper integration can lead to redundancy and failure to achieve the intended learning and assessment outcomes.16
Scenario-based MCQs should meet the ‘cover test’, meaning that if a student knows the answer, he/she will have a key in mind even without seeing the answer options.12,13
EXAMPLES OF SBMCQS
Here are some examples of SBMCQs, which will be valuable for understanding their key elements and structure.
Stem
The stem provides the scenario. The scenario can be based on a clinical case or a concept.
Clinical scenario. The clinical case may begin with the presentation of a problem and followed by relevant signs, symptoms, results of diagnostic studies, initial treatment, and subsequent findings (the relevant components are added, it’s not mandatory to add all clinical details and here we are pointing to the addition of irrelevant clinical details):14,17
Complete sentences
Relevant to the learning objective
Single-themed
Grammatically consistent
Avoid abbreviations and acronyms
Avoid textbook, verbatim phrasing
Nevertheless, it is important to be mindful of window dressing the question redundantly. It involves adding unnecessary details or over-complicating the stem, which can confuse and mislead students. The purpose of the stem is to provide the students with a quick, clear, and accurate picture or idea about what they are asked to answer.
Concept based scenario. SBMCQs do not always require specific patient details or clinical vignettes. While clinical scenarios are commonly associated with SBMCQs, they can be concept-based too.2,18
First and second-year medical students are building foundational knowledge and understanding core concepts. SBMCQs can be designed to assess these without involving detailed patient scenarios.4 It’s important for educators and assessors to recognize the flexibility of SBMCQs and develop these to achieve the assessment goals.
The assessor must not provide any hints to the answer when writing the question stem. Even with limited knowledge of the topic, astute students might deduce the correct answer by spotting repeated words or using grammatical clues such as a/an, is/are, etc. It is crucial to phrase the questions carefully to ensure a fair and unbiased assessment.
Lead-in
The lead-in question should17,18 be (i) unambiguous and straightforward, (ii) provide clear directions to understand the intended objective, (iii) align with the scenario stem and the key (learning outcome), and (iv) avoid the use of negatives, abbreviations, and double negatives, ensuring clarity in the question.
Distractors
These should2,12 (i) be homogeneous in content and style, (ii) be plausible, without any obviously incorrect options, (iii) be similar in construction and length to the correct answer, (iv) be grammatically consistent and related to the stem (avoid clang associations), (v) avoid ‘All of the above’ and ‘None of the above’ options, and (vi) arrange numerical values in a logical sequence.
Key
The key should be clear, unambiguous, and distinct to avoid confusion. It should not consistently appear in a particular position within the list of choices to prevent reliance on position recognition. Ambiguous options can compromise the validity and reliability of the assessment.
By providing a complete reference, you not only ensure the accuracy of the information in the question but also facilitate the review and validation process by making it easier for others to access the source. It should include the following:17 Name of Author(s); Title of the book; Edition; Year of Publication; Publisher; Page number(s)
Allotment of marks
In scoring SBMCQs, it is essential to consider both the difficulty level and the time taken. Negative marking is a contentious matter, requiring a delicate balance to discourage guessing while maintaining a fair and supportive assessment experience for all students. It’s useful in screening tests. However, in formative assessment negative marking should be avoided.
Image-based SBMCQ
Using image-based scenarios in assessments offers a more authentic and contextually rich experience.17,18 Visuals should prompt students to go beyond memorization and lower-order thinking, encouraging analysis and synthesis of information. To enhance higher-level thinking skills, design scenarios with images such as those used in lectures, but with slight variations to challenge students further.
DISCUSSION
SBMCQs in health professional education is useful in the context of the new competency-based medical education (CBME) curriculum and the National Exit Test (NEXT) system. Moreover, SBMCQs can assess the analytical and problem-solving capabilities of the examinees. However, designing such questions is challenging for developers and requires proper planning along with appropriate training. In the absence of proper training, the process can lead to redundant questions or incomplete stem or ambiguous distractors.2
Most teachers of basic medical sciences are aware of and have good knowledge of SBMCQs, but regular training and workshops are recommended for practicing the same.20
Using the right tool, in the right context, at the right juncture, supplemented by other tools, and backed by constructive feedback, can help nurture the good intent ingrained in the CMBE curriculum for the development of an assessment toolbox for judging the competencies of Indian medical graduates.5
The major challenge in designing MCQs are lack of training in construction, most were prepared as context-free in the last minutes, with minimal process of vetting or review the quality of questions and lack of any standardized question format, lack of accountability and underestimation of the use of blueprint of assessment by medical teachers. It is important to use a realistic clinical setting, routinely practiced, and relevant clinical findings in assessment to make the student demonstrate the intended knowledge or skill. However, without attention to detail, the scenario in a clinical science question will only lead to frustration among candidates, reading of unnecessary materials and threaten the reputation of the assessment.14,16
In the present conditions where competitive examinations are on the rise and the assessment process is getting more inclined to SBMCQs, increasing the number of scenario questions will certainly mean the inclusion of wider course content.20
Conclusion
Well-designed SBMCQs can promote critical thinking and revolutionize the way we assess knowledge. Therefore, it is essential to train faculty in constructing meaningful questions. By using SBMCQs as a means of assessing students, educational institutions can establish a comprehensive evaluation framework that combines efficiency and efficacy. This approach enhances the student’s learning experience and contributes to improvement of the educational process.
ACKNOWLEDGEMENTS
We wish to express our sincere appreciation for the invaluable contributions made by the fellows, advisors, and esteemed faculty members of CMCL FAIMER during the discussion on the topic of ‘Designing of scenario-based questions’. Their collective efforts have enriched the depth and quality of our work. The inputs they provided played a pivotal role in shaping this article into its current form.
Conflicts of interest.
None declared
References
- Competency-based undergraduate curriculum for the Indian Medical Graduate; 2018:1-3. Available at www.mciindia.org/CMS/information-desk/for-colleges/ug-curriculum (accessed on 5 May 2023).
- [Google Scholar]
- Triple Cs of scenario-based multiple-choice question: Concept, construction, and corroboration. Natl J Pharmacol Ther. 2023;1:8-12.
- [Google Scholar]
- Pushing critical thinking skills with multiple-choice questions: Does Bloom's taxonomy work? Acad Med. 2018;93:856-9.
- [CrossRef] [PubMed] [Google Scholar]
- Multiple case scenarios based on integrated teaching among first year medical students: A cross-sectional study. J Clin Diagn Res. 2021;15:1-5.
- [CrossRef] [Google Scholar]
- Assessment toolbox for Indian medical graduate competencies. J Postgrad Med. 2021;67:80-90.
- [CrossRef] [PubMed] [Google Scholar]
- Regulations on Graduate Medical Education (Amendment) Addition as Part-II for MBBS course starting from academic year 2019-20 onwards 2019. Available at www.nmc.org.in/ActivitiWebClient/open/getDocument?path=/Documents/Public/Portal/Gazette/GME-06.11.2019.pdf (accessed on 1 Jul 2023).
- [Google Scholar]
- Previous years' question papers. Available at www.bfuhs.ac.in/examination/oldquespapers/MJ14ND14MJ15ND15/MJ14ND14MJ15ND15.HTML (accessed on 1 Jul 2023)
- [Google Scholar]
- Exam moderately difficult says expert. Available at www.indiatoday.in/education-today/news/story/neet-pg-2023-analysis-exam-moderately-difficult-says-expert-2342888-2023-03-05 (accessed on 1 Jul 2023)
- [Google Scholar]
- Examining Bloom's taxonomy in multiple choice questions: Students' approach to questions. Med Sci Educ. 2021;31:1311-17.
- [CrossRef] [PubMed] [Google Scholar]
- Incorporation of Bloom's taxonomy into multiple-choice examination questions for a pharmacotherapeutics course. Am J Pharm Educ. 2012;76:114.
- [CrossRef] [PubMed] [Google Scholar]
- Effect of faculty training on quality of multiple-choice questions. Int J Appl Basic Med Res. 2020;10:210-14.
- [CrossRef] [PubMed] [Google Scholar]
- Guidelines for the construction of multiple-choice questions tests. J Family Community Med. 2006;13:125-33.
- [CrossRef] [PubMed] [Google Scholar]
- Framing a well-structured single best response multiple choice question (MCQs)--An art to be learned by a teacher. J Univ Coll Med Sci. 2014;2:60-4.
- [CrossRef] [Google Scholar]
- Writing clinical scenarios for clinical science questions. Clin Med. 2016;16:142-5.
- [CrossRef] [PubMed] [Google Scholar]
- How to write a high-quality multiple-choice question (MCQ): A guide for clinicians. Eur J Vasc Endovasc Surg. 2017;54:654-8.
- [CrossRef] [PubMed] [Google Scholar]
- Multiple choice questions in medical education: How to construct high-quality questions. Int J Hum Health Sci. 2020;4:79-88.
- [CrossRef] [Google Scholar]
- Principles of medical education: A review of book's fifth edition In: Int J Appl Basic Med Res. Vol 11. 2021. p. :125-7.
- [CrossRef] [Google Scholar]
- Constructing written test for the basic and clinical sciences. 2002. (3rd ed). Philadelphia, PA, USA: National Board of Medical Examiners; Available at www.nbme.org/about/itemwriting.asp (accessed on 1 Jul 2023)
- [Google Scholar]
- Davidson's Principles and practice of medicine. (23rd ed). London: Elsevier; 2018. p. :318.
- [Google Scholar]
- Scenario-based, single best, multiple-choice questions (SB-SB-MCQs) in basic medical sciences: An exploratory study about the staff awareness, knowledge and difficulties encountered. J Biosci Med. 2022;10:79-85.
- [CrossRef] [Google Scholar]
