Assessment Foundations for Effective Reading Instruction

Foundations of Assessment Competency 3 Session 3 Acknowledgements: These materials were prepared by the Florida Center for Reading Research in partnership with the Florida Department of Education, the Just Read Florida! Office, and faculty in the College of Education at Florida State University. The authors would especially like to thank the following faculty and staff for their significant contributions to creating and reviewing materials for this integrated reading endorsement pathway: Mr. Nathan Archer, Ms. Amy Carroll, Dr. Jennifer Gans, Dr. Jennifer Hamilton, Dr. Laurie Lee, Dr. Arzu Leushuis, Ms. Shayla Lightfoot-Brown, Dr. Nicole Patton Terry, Dr. Kevin Smith, and Dr. Kelly Whalon. We acknowledge the authors of the Professional Learning Community materials that support the Foundational Skills to Support Reading for Understanding in Kindergarten Through 3rd Grade*. With those authors' permission, as well as additional resources and materials developed by instructional leaders in Seminole County Public Schools, the same format and five step process for implementing the PLC sessions were utilized for content in Competencies 1 and 2. Permission to reprint or use these materials is required. Inquiries may directed to the Florida Center for Reading Research at fcrr@fcrr.org. *Kosanovich, M. & Foorman, B. (2016). Professional learning communities facilitator s guide for the What Works Clearinghouse practice guide: Foundational skills to support reading for understanding in kindergarten through 3rd grade (REL 2016-227). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Southeast. Retrieved from http://ies.ed.gov/ncee/edlabs.

Purpose of this Module This course provides a foundation in assessment with an emphasis on literacy/reading and is required for the Florida Reading Endorsement (Competencies 3) Grounded in the principles of research-based reading instruction and the Reading Endorsement Guiding principle that teaching reading for understanding is an ongoing systematic, problem solving process, teachers will implement and analyze assessments, and select appropriate instruction/interventions based on the collected data.

Foundations of Assessment Part 2

Technical Adequacy Validity A valid test measures what it intends to measure Allows us to make valid inferences about test data Reliability A reliable test provides consistent results Allows us to generalize a person s test performance to other tests items measuring the same construct/domain, a different day/time, and with different testers Let s Learn More Why are reliability and validity important considerations when assessing reading? Explain to the person next to you.

All Tests Include Error Standard Error of Measurement (SEM) Because we are unable to measure performance across all possible times, items, testers, there is always error Every score includes the learner s true score + error SEM tells us how much error or noise there is in a test score The larger the SEM the greater the uncertainty of a test score SEM is used to help us interpret test scores quantifying accuracy of the score SEM is used to create confidence intervals

Confidence Intervals Confidence intervals are a range within which the true score has a known probability of falling A confidence interval is the score +/- the SEM If a score is 100 and the SEM is 5, then the confidence interval is +/-5 The confidence interval is based on how certain we want to be that the true score falls within a specified range Levels of confidence typically reported are 68, 90 and 95% When making important decisions we want to be confident 95% We select the desired level of confidence first. For example, If we want to be 95% confident in a vocabulary knowledge test score and a child scored a 99 with a 95% confidence interval of +/-4, we would say that we are 95% confident that the child s true score falls between 95-103 If our confidence interval is really large, that will mean error is large

Your Turn to Interpret Tom s standard scores with confidence interval (95%) on the Test of Preschool Early Literacy (TOPEL) Phonological Awareness 96 +/- 7 (89-103) Print Knowledge 99 +/- 5 (94-104) Definitional Vocabulary 78 +/- 6 (72-84) Explain in your own words

Bias All tests have error Bias can occur when We do not consider who the test is designed for and for what purpose Background knowledge and experiences of the learner Failure to follow standardized procedures

Relationship Between Reliability and Validity Anything that interferes with a test measuring what it is intended to measure affects validity Therefore, if a test is not reliable it is not valid Reliability, norm groups, bias, administration, interpretation, error, etc. are all subparts of validity Describe the relationship in your own words. Consider this scenario Lupe has lived in the US for 3 months. When selecting a norm-referenced test to determine her current reading level, what are some considerations for reliability and validity.

Important Criteria for Establishing Validity A test must be reliable to be valid, but reliability does not ensure validity Students must have the skills necessary to complete the test Language Hearing Items should work the same way for various groups of students Administration errors Representative norms

Considerations for English Language Learners Marco is in the 2nd grade and recently moved to the United States from Mexico. He loves soccer, music and speaks fluent Spanish. Marco has had little education in the English language. He knows some basic, functional words and phrases and can follow simple instructions in English. Marco has had little instruction in phonics and has trouble reading and spelling. He is better at speaking than reading and writing. It is clear to his teachers that Marco is bright. He is very interested in mathematics and science and when his books are translated he clearly understands and applies the content. What are some important considerations when selecting assessments for Marco? (consider reliability and validity) What informal and formal assessments would you recommend to determine Marco s current reading levels? How should this data be used to support Marco s reading proficiency in English? What would your recommendation be to monitor progress?

References and Resources for Foundations of Assessment Helman, L., Ittner, A. C., & McMaster. K. (2019). Assessing language and literacy with bilingual students: Practices to support English learners. New York, NY: Guilford Press. Salvia, J., Ysseldyke, J., & Witmer, S. (2017). Assessment in special and inclusive education (13th ed.). Boston, MA: Cengage Learning. Stahl, K. A. D., Flanigan, K., & McKenna, M. C. (2019) Assessment for reading instruction (4th ed). New York, NY: Guilford Press.

Additional Web Sources Not Embedded in Slides International Dyslexia Association. (2017). Dyslexia assessment: What is it and how can it help? https://dyslexiaida.org/dyslexia-assessment-what-is-it-and-how-can-it- help/ Psychology Crash Course: Brains vs. Bias https://www.youtube.com/watch?v=75g4d5sF3xI Edmentum: Breaking Down Standard Error of Measurement https://www.youtube.com/watch?v=FdZIIGMINmw