Effective Strategies for Crafting Quality Test Questions
Discover the key components of a well-constructed multiple-choice question, including distractor power, item difficulty, and item discrimination. Learn how to assess question quality and ensure validity in assessments.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Item Analysis What makes a question good??? Answer options?
Multiple Choice Questions Elements of a good multiple choice question????
Item Analysis How can we determine if it is a good question????? Distractor power Item difficulty Item discrimination
Distractor power Strength of each distractor (should be equal) # answering incorrect # of distractors
E.G. There are 4 possible answers Option a is correct if 9 people choose option b (9/3 = 3) and 12 people choose option c (12/3 = 4) and 15 people choose option d (15/3 = 5) then option d is the strongest distractor
Item Difficulty = p (percent of people passing) p = number answering correctly total number answering If 100 people take the test and 34 get question 1 correct, what is the difficulty of that question? p1 = 34/100 = .34 For 65 correct p would be 65/100 = .65 SO the higher the p, the easier the question (score must be between 0 and 1)
Item Difficulty is... A behavioral measure Based on the current group Allows internal comparison Also allows external comparison
In construct based tests Item difficulty is similar to item endorsement Number agreeing with an item
Item endorsement also... A behavioral measure Based on the current group Allows internal comparison Also allows external comparison
Item discrimination = d Does the question discriminate between those who do well overall and those who do poorly? (or on a psychological test, those scoring high or low on the construct)
Distribution To determine level upper (U) = top 25 to 33 % of scores lower (L) = bottom 25 to 33 % of scores
Determining item discrimination d = U-L n where d = item discrimination U = number in the upper group who get question correct L = number in the lower group who get question correct n = number in one group (U or L)
Examples of d With 33 in U and L if all in U and all in L get it correct 33 - 33 / 33 = 0 if all in U get it correct, and none in L 33 - 0 / 33 = 1 if none in U get it correct, and all in L 0 - 33 / 33 = -1 if 30 in U and 10 in L get it correct 30 - 10 / 33 = .61
Value of d Range for d = -1 to 1 higher d indicates more discriminating (1 = all in upper group got it correct and none in lower group) negative d indicates a really bad question (-1 indicates all in lower group got it correct and none in upper group)
