Writing Effective MCQs - Marc J. Kahn, MD, MBA - Guidelines for MCQ Creation
This presentation covers the essential elements of creating effective Multiple Choice Questions (MCQs) for educational assessments. It includes tips on question types, pitfalls to avoid, psychometrics, and MCQ rules, ensuring the development of high-quality assessment tools to evaluate knowledge and comprehension. The content delves into MCQ anatomy, types, and optimizing question construction for accurate assessment outcomes.
Uploaded on Feb 21, 2025 | 0 Views
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Writing Effective MCQs Marc J Kahn, MD, MBA Peterman-Prosser Professor Sr. Associate Dean Tulane University School of Medicine
Disclosures Nothing to disclose
There are no perfect questions, only perfect answers
MCQ Types A through R and X USMLE Step 1 only uses A-type, Step 2 also uses R- type A-type = single best answer MCQ, 4 or more options R-type = extended matching in sets of 2-20
K-type Hyperleuckocytosis is typically associated with (1) infection (2) acute leukemia (3) chronic leukemia (4) baseball A=1,2,3 B=1,3 C=2,4 D=4 E=all
Problem with multiple T/F 1 2 3 4 Totally Wrong Totally Correct
Problem with multiple T/F 1 3 4 2 Totally Wrong Totally Correct
Psychometrics How precise is my test? Kuder Richardson tests reliability for measures with dichotomous choices. Ranges 0.00 to 1.00 KR=[N/(N-1)][1-{M(N-M)}]/(N*V) N= number of items M=arithmetic means of test scores V= variance of raw scores > 0.7 is good
Point Biserial Measures correlation of individual question with overall examinee performance How good is my question at distinguishing high from low performers? rpb=M1-M0/sn-1 * [n1n0/n(n-1)]^1/2 Want POSITIVE value (-1.00 to 1.00) >0.2 very good
MCQ rules Focus on potentially catastrophic or common problems Assess application of knowledge not a factoid Stem should be clear All distractors should be homogenous Avoid technical flaws
Anatomy of a MCQ STEM Lead in A. Distractor B. Distractor C. Correct Answer D. Distractor E. Distractor
Bad Stem Beta blockers are associated with what common side effect?
Better stem A 67 year old woman complains of one month duration of severe shortness of breath associated with a dry cough. Her symptoms are worse with exercise. She has been diagnosed with severe glaucoma that is being treated with timolol, dipivefrine, travaprost, and brimonidine eye drops. She is also taking oral acetazolamide. Which of her medications is most likely responsible for her symptoms?
Other STEM problems Too much information Unnecessary data Red herrings Should all follow same format: Age and gender Complaint and duration PMH Physical Labs
Lead-in problems Multiple negatives All of the following are not contraindications to beta blockers except for: Disguised TRUE/FALSE Which of the following are true about his condition?
A glib is purveyed Which of the following are also purveyed: A. Binks B. Astons C. Glabs D. Wims E. Creys
The perils of all and none of the above A. mitochondria B. endoplasmic reticulum C. Nucleus D. None of the above E. All of the above
Schrodingers Cat Dead Alive Or in between?
Bad Distractor A. red blood cell B. lymphocyte C. neutrophil D. eosinophil E. basophil
Universality A. Prevalence is highest in childhood B. Prevalence is highest in adulthood C. Prevalence is highest in the elderly D. Is autosomal dominant E. Prevalence is highest in women
Non-homogeneity A. review peripheral smear B. give IV glucose C. discuss end of life care D. Consult a cardiologist E. order a blood gas
Other key points Perform psychometrics Keep a secure item bank Add and retire questions (~30% new questions per year) Share with peers for comment Exam item Review Committee Insure clinical relevance
It is better to give than to receive ???
