Measuring Reading Fluency: Response Time vs. Accuracy
Explore a study on measuring reading fluency through the combination of response time and accuracy. The research aims to incorporate both factors into fluency assessment, presenting a new model for recoding responses based on response time and accuracy. By comparing different models, this study offers insights into enhancing reading fluency evaluation.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Measuring Reading Fluency through the Combination of Response Time and Accuracy Mingfeng Xue; Qing Cai University of California, Berkeley mingfengxue@berkeley.edu
Introduction Reading fluency is a part of the PISA reading literacy framework It is defined as the ease and efficiency with which one can read and understand a PISA 2018 reading literacy framework Source: OECD(2019) piece of text Name of presentation | date
Introduction PISA measures reading fluency via 22 sensical and non-sensical sentences within 3 mins PISA 2018 mainly focuses on response time An example of PISA 2018 reading fluency item Name of presentation | date
The present study Aims to incorporate both response time and accuracy into measuring reading fluency Proposes to recode responses based on response time and accuracy The joint hierarchical model for response accuracy and time Compares it with the hierarchical Name of presentation | date model for response accuracy and time (van der Linden, 2007)
The present study New response Accuracy Time 4 correct fast 3 correct slow 2 incorrect fast 1 incorrect slow Note. Fast means the response time is less than the median response time; slow means the response time is larger than the median response time Name of presentation | date
Methods 65 items in the sentence judgment test of PISA 2018 4,786 US students Apply the hierarchical model (van der Linden, 2007) to simultaneously model response time and accuracy Apply the Generalized Partial Credit Model (GPCM) to Name of presentation | date recoded responses
Methods Evaluation criterion: reliability and validity Reliability: Expected a posteriori estimates Validity: convergence validity criterion validity reading literacy score (RL) locating information (RLLI) understanding (RLUN) evaluating and reflecting (RLER) Name of presentation | date
Results Estimates EAP Reliability Hierarchical model Ability 0.494 Speed 0.897 Recoded responses and GPCM New estimate 0.852 Our new estimates demonstrate that satisfactory reliability Name of presentation | date
Results Correlations among latent traits and speed estimated from LNIRT and estimates from our new methods Ability Speed New estimate Ability Speed .107* New estimate .600** .741** Our new estimates are moderately correlated with the ability and speed estimates from the hierarchical model Name of presentation | date *<.05; **p<.01;
Results Correlations coefficients between the three reading fluency estimates with reading scores from PISA Hierarchical Model New estimate Ability Speed Reading literacy score .540** .329** .556** Locating information .452** .341** .517** Understanding .464** .347** .530** Evaluating and reflecting Our new estimates show highest association with the reading scores from PISA reading assessment .454** .350** .526** Name of presentation | date
Conclusion Response time and accuracy are two inseparable characteristics of reading fluency Recoded responses based on response time and accuracy can be a useful method to measure reading fluency Name of presentation | date
Thank you Together we build a better future of measurements Name of presentation | date
References OECD (2019b), PISA 2018 Assessment and Analytical Framework, PISA, OECD Publishing, Paris. van der Linden, W. J. (2007). A hierarchical framework for modeling speed and accuracy on test items. Psychometrika, 72(3), 287. Name of presentation | date
