Understanding Holistic Scoring in Education
Learn about the importance and history of holistic scoring in education, its benefits, and how it is used in assessing the overall quality of student work. Explore the shift from TEAC to CAEP standards and the collaborative approach required for effective evaluation.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
WHO WE ARE https://www.youtube.com/watch?v=usQem53fDH8
A BIT MORE BACKGROUND http://www.albion.edu/academics/departments/education http://www.facebook.com/Albion.College.Teacher.Education Approximately 1300 total enrollment; 80 in TEP TEAC accredited through October 1, 2016
A BIT MORE BACKGROUND (contd) Elementary certifications: English language arts, integrated science, mathematics and social studies Secondary certifications: biology, chemistry, earth/space science (geology), English, French, German, history, mathematics, physics, political science, psychology, social studies (major only) and Spanish K-12 certifications: French, German, Spanish and music
WHY HOLISTIC SCORING? Teacher Education Accreditation Council (TEAC) accreditation weakness change from Teacher Education Accreditation Council (TEAC) national standards to Council for the Accreditation of Educator Preparation (CAEP) standards need to identify what would constitute evidence and a tool for assessing the evidence financial constraints requires collaboration
HISTORY OF HOLISTIC SCORING Gray, J. (1982). Properties of writing tasks: A study of alternative procedures for holistic writing assessment. final report. http://eric.ed.gov/?id=ED230576 Hunter, D. M. (1996). The use of holistic versus analytic scoring for large-scale assessment of writing.http://eric.ed.gov/?id=EJ543798
WHAT IS HOLISTIC SCORING? Holistic scoring is a process for assessing and grading the summative quality based on professional and personal knowledge, the particular process and event, the purposes at hand, and the meanings shared by the individuals involved. An overall impression includes everything...; this cannot be broken into component parts, per se. A holistic system for scoring is legitimate and reasonable because it requires the scorers to cast a comprehensive view .... The process is synergistic because it requires iterative thinking by each participant, and invokes collaboration. It is a kind of gestalt; the whole is greater than the sum of its parts. (after Dr. Judy Nichols Mitchell)
WHAT IS HOLISTIC SCORING (contd)? Advantages Focuses on what writers can do, and actually did, rather than on what they didn t do; Links function, content, and form-the acts of composing and producing; Serves as an evaluation measure and/or needs assessment; and, Is authentic to the demands of real writing tasks.
WHAT IS HOLISTIC SCORING (contd)? Preparation Compose/select prompt Training Describe system Application Read and score all materials Explain prompt & anchor trial Collect materials Sort/tabulate all scores Select anchor set Tally/discuss scores Discuss and react Select scoring scale Develop/confirm criteria
WHAT IS HOLISTIC SCORING (contd)? Reference for four-point scoring scale Three categories (unsatisfactory/low; basic/middle; and proficient/high); Distinguished category eliminated due to DFFT/EI conceptual framework for this category; Round down on split scores (e.g., 0+1 or 2+3); Non-adjacent and non-equivalent scores (e.g., 1, 3; 2, 4) require third scoring; and, Across three scores, count two lowest scores. See also Danielson Framework for Teaching/Evaluation Instrument.
WHAT IS HOLISTIC SCORING (contd)? Scoring Scale 0 ( unsatisfactory) 1, 2 ( basic) 3, 4 ( proficient) 0, 1 ( unsatisfactory) 2, 3, 4, 5 ( basic) 6, 7, 8 ( proficient)
WHAT IS HOLISTIC SCORING (contd)? SAMPLE ANCHOR SET SCORING TABLE 4 3 2 1 0 A B C
WHAT DID WE ACHIEVE? TEP Curriculum Map Danielson Framework for Teaching Evaluation Instrument CAEP Standard 5
TEP Curriculum Map https://www.dropbox.com/s/q4q5lazfnktzoon/Converted%20TE P%20%26%20Danielson%20FFT%20Evaluation%20Instrumen t_correlation_v2.1.xlsx?dl=0
CAEP Standard 5: Provider Quality Assurance & Continuous Improvement The provider maintains a quality assurance system comprised of valid data from multiple measures, including evidence of candidates and completers positive impact on P-12 student learning and development. The provider supports continuous improvement that is sustained and evidence-based, and that evaluates the effectiveness of its completers. The provider uses the results of inquiry and data collection to establish priorities, enhance program elements and capacity, and test innovations to improve completers impact on P-12 student learning and development.
CAEP Standard 5: Provider Quality Assurance & Continuous Improvement (cont d) Quality and Strategic Evaluation 5.1 The provider s quality assurance system is comprised of multiple measures that can monitor candidate progress, completer achievements, and provider operational effectiveness. Evidence demonstrates that the provider satisfies all CAEP standards. 5.2 The provider s quality assurance system relies on relevant, verifiable, representative, cumulative and actionable measures, and produces empirical evidence that interpretations of data are valid and consistent.
CAEP Standard 5: Provider Quality Assurance & Continuous Improvement (cont d) Continuous Improvement 5.3. The provider regularly and systematically assesses performance against its goals and relevant standards, tracks results over time, tests innovations and the effects of selection criteria on subsequent progress and completion, and uses results to improve program elements and processes...
WHAT DID WE ACHIEVE (contd)? Field Experience & Student Teaching Feedback Forms Lesson Plan Form reflection (e.g., Time I Heard Self)
FIELD EXPERIENCE FEEDBACK FORM Old Form com/s/nv8yc7s72ob86p2/ASSESSMENT%20for%20NEW%20PRO GRAM%20updated%20Summer%202007.doc?dl=0 New Form https://www.dropbox.com/s/jr1adctce07a5oa/field%20experie nce_final%20evaluation__tier%201%5BR%5D.docx?dl=0
STUDENT TEACHING FEEDBACK FORM Old Form https://www.dropbox.com/s/hd297ic7jqc2gk7/final- feedback%20form%20for%20mentor%20teachers.docx?dl=0 New Form https://www.dropbox.com/s/82mlhomshig2si7/field%20experie nce_final%20evaluation%5Bmentor_teachers%5D__tier%204 .docx?dl=0
WHAT PROBLEMS DID WE HAVE? Mental shift from scoring on separate attributes to scoring artifact as a whole. Review one artifact in relation to one DFFT element; even if two might be applicable, cumbersome to score. Explain the context of the artifact course, assignment, and objectives of assignment. As we progressed in our understanding of the DFFT, we re- assigned artifacts to different elements. Can t do it all small program, so we can score artifacts from all students without sampling; we can t score all artifacts from all students. Choose for coverage of the framework.
WHATS NEXT? Add more tiers to our process. Deepen our understanding of the DFFT to improve the validity of artifacts-to-elements match. Expand to actual pre-service teacher performance.
