Writing the Test Blueprint - Module 7 Overview
In this module, learn about test blueprints, their components, and criteria for a good blueprint. Discover the importance of assessment in education and the purpose it serves in teaching and learning processes.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Michigan Assessment Michigan Assessment Consortium Consortium Common Assessment Common Assessment Development Series Development Series Module 7 Module 7 Writing the Test Blueprint Writing the Test Blueprint 1
Narrated By: Narrated By: Bruce Fay Wayne RESA 2
In This Module, You Will In This Module, You Will Learn About: Learn About: Test blueprints what they are and why you need them The components of a test blueprint Criteria for a good test blueprint Test blueprint example 3
If you don't know where If you don't know where you're going, any road will you're going, any road will take you there. take you there. George Harrison (1943 - 2001) "Any Road", Brainwashed, 2002 Adapted from Lewis Carroll
Assessment with a Purpose Assessment with a Purpose Educational assessment is not something incidental to teaching and learning. It is an equal partner with curriculum and instruction. It is the critical 3rdleg through which both students and teachers receive feedback about the effectiveness of the teaching and learning process in achieving desired learning outcomes. Assessment closes the loop. 5
Closed Closed Loop Systems Loop Systems (Feedback) (Feedback) Actual Temperature (Test Results) Desired Temperature (Learning Target) Home Heating System (Teaching & Learning)
Purposeful Assessment Purposeful Assessment Requires thoughtful alignment ensuring that the items on a test fairly represent the intended (curriculum), and actual (instructional) learning targets 7
Common (shared) Common (shared) Assessment Assessment Used by multiple teachers Used multiple times Based on agreed upon learning targets C I A Test what you teach, teach what you test 8
Assessing Learning Targets Assessing Learning Targets Relative importance of targets Level of cognitive complexity associated with the targets 9
End Result End Result Creating tests that are useful requires them to be Reliable Free of bias Fair Used in valid ways Close the loop 10
What is a Test Blueprint? What is a Test Blueprint? Plan for constructing a common assessment Learning targets Levels of complexity Importance Common assessments work best within a collegial process 12
Learning Targets Learning Targets State and national standards Local decisions What to emphasize? How to teach and assess? 13
Cognitive Complexity Cognitive Complexity Norm Webb (1997) Depth of Knowledge Recall (level 1) Skill or concept use or application (level 2) Strategic thinking (level 3) Extended thinking (level 4) First task clarify level of complexity of the content 14
Basic Example with Basic Example with 5 Targets and 3 Levels 5 Targets and 3 Levels Webb (1997) Depth of Knowledge Target Totals Recall Use Strategic Extended # Points # Points # Points # Points # Points Target 1 3 3 2 2 5 5 Learning Target Codes Target 2 1 1 2 2 2 4 5 7 Target 3 2 2 1 3 3 5 Target 4 3 3 1 2 4 5 Target 5 2 4 1 4 3 8 DOK Totals 6 6 10 14 4 10 0 0 20 30 15
Basic Example with Basic Example with 5 Targets and 3 Levels 5 Targets and 3 Levels Webb (1997) Depth of Knowledge Recall Use Strategic Extended Target Totals # Points # Points # Points # Points # Points Target 1 3 3 2 2 5 5 Learning Target Codes Target 2 1 1 2 2 2 4 5 7 Target 3 2 2 1 3 3 5 Target 4 3 3 1 2 4 5 Target 5 2 4 1 4 3 8 DOK Totals 6 6 10 14 4 10 0 0 20 30 16
20 item test for 5 targets and 20 item test for 5 targets and 3 levels worth 30 points total 3 levels worth 30 points total Webb (1997) Depth of Knowledge Recall Use Strategic Extended Target Totals # Points # Points # Points # Points # Points Target 1 3 3 2 2 5 5 Learning Target Codes Target 2 1 1 2 2 2 4 5 7 Target 3 2 2 1 3 3 5 Target 4 3 3 1 2 4 5 Target 5 2 4 1 4 3 8 DOK Totals 6 6 10 14 4 10 0 0 20 30 17
Is This Reasonable? Is This Reasonable? Rule of Thumb Criteria Rule of Thumb Criteria At least 3 items per target (5 is better) for reliability Appropriate distribution of items over targets (1 & 2 have more items, but fewer points than 3 & 5) Levels of complexity (depth of knowledge) are appropriate for targets and instruction Appropriate distribution of items over levels of complexity (All items are NOT at the lowest or highest level) 18
Professional Judgment Professional Judgment Yes! Improves with experience Collective decisions Reflection regarding methods and results 19
Basic Example Recap Basic Example Recap Webb (1997) Depth of Knowledge Recall Use Strategic Extended Target Totals # Points # Points # Points # Points # Points Target 1 3 3 2 2 5 5 Learning Target Codes Target 2 1 1 2 2 2 4 5 7 Target 3 2 2 1 3 3 5 Target 4 3 3 1 2 4 5 Target 5 2 4 1 4 3 8 DOK Totals 6 6 10 14 4 10 0 0 20 30 20
Some Added Sophistication Some Added Sophistication (keeping track of item types) (keeping track of item types) Appropriate match to learning targets and associated levels of complexity Balanced use within tests and across tests over time Other test related materials Same or separate spreadsheet 21
Common Item Types Common Item Types Selected-response (multiple-choice) Constructed-response (fill-in-the-blank or short answer) Extended-response (outline, essay, etc.) Matching Sort/arrange a list in order Performance tasks Projects 22
Item Type Depth of Knowledge Level Row Totals S-R C-R C-R Perf (cell entries are points) (points per) Brief Extended Recall Use Strategic Extended Item Target 1.1.1.1 x 1 1 Learning Target Codes 1.1.1.1 x 1 1 1.1.1.1 x 2 2 4 1.2.3.4 x 3 3 1.2.3.4 x 5 5 8 etc. Column Totals 2 2 1 0 1 6 5 0 # of point s Item & point totals # of items 5 12 24
Complete Test Specification Complete Test Specification Basic Learning Targets with number of items and points for each Intermediate Explicit representation of items, item types, and points for each Learning Target Make it work for you. 25
Summary Summary Purpose Establish parameters Outcomes Learning targets to be measured Number and type of items Explicit map / shared document 26
Summary Summary Professional discussion and decisions Common means shared Purpose Effort Use Understanding The Test Blueprint Simple but critical to your success 27
Series Developers Series Developers Kathy Dewsbury White, Ingham ISD Bruce Fay, Wayne RESA Jim Gullen, Oakland Schools Julie McDaniel, Oakland Schools Edward Roeber, MSU Ellen Vorenkamp, Wayne RESA Kim Young, Ionia County ISD/MDE 28
Development Support Development Support The MAC Common Assessment Development Series is funded in part by the Michigan Association of Intermediate School Administrators In cooperation with Michigan Department of Education Ingham and Ionia ISDs, Oakland Schools, and Wayne RESA Michigan State University 29
