Research Methods in Education & Social Work
Delve into the world of research methods in education and social work with a focus on repeated measures, longitudinal studies, and analyzing data to understand changes over time. Explore how to measure and interpret data, conduct paired t-tests, and evaluate differences in gain scores between groups. Uncover the complexities of data structures and predictive analyses across multiple time points, guided by insights from Prof. Gavin Brown.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Prof. Gavin Brown 2020 PG Research Methods School, Faculty of Education & Social Work
Time Number of waves of data 2 times allows change in repeated measures (Linked pairs) Longitudinal expects 3 or more waves Metric for time Type of unit (minutes, days, weeks, months, semesters, sessions, etc.) Fixed schedule vs. flexible timing Change Often= growth, improvement; esp. in 2 wave studies; Complex trajectories likely (e.g., rise, plateau, decrease, rise); need 3 waves min.
Focus of interest Level 1: change in individuals Level 2: external factors that influence Level 1 changes Measurements at 1 time will be more alike than those between times, so time can be a clustering factor Repeated Measures are similar because they are repeated Time 1 predicts Time 2, even if there are in-between things; sometimes just because things are good predictors of themselves later on: Thus Correlated across time at least Autoregressive in path or regression analysis Data structure Manifest variable vs. latent structures
Repeated measures (2 times) O1 Stuff happens between times; sometimes planned as experiment Evaluation of change in score between T1 and T2 Longitudinal study (3 or more times) O1 Change is not necessarily linear (quadratic, cubic, exponential, etc.) Complex analysis methods needed More complex if using latent MIMIC structures instead of manifest variables - - stuff over time - - - - O2 - O2 O3 - O4 - O5 O6 ..
Time 2 score minus Time 1 Score di= y1i y2i difference score is unbiased estimate of true change How to tell if diis beyond chance? Paired t-test (esp. if N<30) Otherwise ANOVA Convert to effect size How do groups, on average, differ in gains? Conduct a one-way ANOVA of gain scores Simple and appropriate for telling if O2is > O1by more than chance Dependent = gain score Predictor = group 1 group 2 times 2 groups, 2 times
t-test of gain scores = paired t-tests paired t-test is the difference between "each pair of observations , which are gain scores t tests for correlated data (or matched pairs, or repeated measurements), perform our calculations on differences between the n1and n2scores
Gain Score Matched t-test
Using Microsoft Excel file (repeated measures 2 times data.xls) Open comments to see instructions Calculate gain differences Time 2- Time 1 Calculate M, SD for 2 groups at each time Calculate gain M, SD for each group Calculate correlation of T1 to T2 scores for each group Calculate effect size using online effect size calculator at https://campbellcollaboration.org/research-resources/effect-size- calculator.html Use multiple procedures are the results identical?
Sewall Wright Wright, S. (1921). Correlation and causation. Journal of Agricultural Research, 20: 557 585. Wright, S. (1934). Physiological and evolutionary theories of dominance. The American Naturalist, 68:714 24-53 describes the directed dependencies among a set of variables a special case of SEM in which only single indicators are employed for the variables in the causal model. Does regression analysis PLUS keeps Residuals Correlations Can add mediating and moderating variables But relies on manifest variables May be useful for SMALL sample size problems
The rules for path tracing are: You can trace backward up an arrow and then forward along the next, or forwards from one variable to the other, but never forward and then back. You can pass through each variable only once in a given chain of paths. No more than one bi-directional arrow can be included in each path-chain. The model has no feedback loops: the directed graph of the model must contain no cycles (i.e. it is a directed acyclic graph; see Judea Pearl on DAG)
NB. Correlated residuals at start to show the 2 variable chains are correlated Bivariate with cross-lagging
This is a time based repeated measure Design: Bivariate with cross-lagging Manifest variables Residuals Correlations Moderation Good with low N training Follow up Brown, G. T. L., & Marshall, J. C. (2012). The impact of training students how to write introductions for academic essays: An exploratory, longitudinal study. Assessment & Evaluation in Higher Education, 37(6), 653-670. doi:10.1080/02602938.2011.563277
Open SPSS path data.sav Delete all persons without complete data, save as path data2.sav Open AMOS path model exercise.amw Link Model to path data2.sav Add autoregressive and cross-lag paths Add exogenous correlation of pre-intervention style & structure/content Run model; save Open results and compare to published result
Complex models can have multiple correlated predictors multi-collinearity means not all predictors have a statistically significant path to dependent variables Multiple regression with step-wise or hierarchical approach gets close but only with manifest variables But SEM can incorporate: Links Measurement models to structural models Full latent variables correlated predictors, outcomes, and mediators AND Unexplained variances or residuals on manifest and latent variables Moderation & mediation relationships You still need a theoretical causal explanation
Harper, A., & Brown, G. T. L. (2017). Students use of online feedback in a first year tertiary biology course. Assessment Matters, 11, 99-121. doi:10.18296/am.0026 What do the path values tell you about the people who used the system?
2 times we assume change is linear With multiple time points change is not necessarily linear Peterson, E. R., Brown, G. T. L., & Jun, M. C. (2015). Achievement emotions in higher education: A diary study exploring emotions across an assessment event. Contemporary Educational Psychology, 42, 82-96. doi:10.1016/j.cedpsych.2015.05.002 Brown, G. T. L., & Peterson, E. R. (2018). Evaluating repeated diary study responses: Latent curve modeling. In SAGE Research Methods Cases Part 2. Retrieved from http://methods.sagepub.com/case/evaluating-repeated-diary- study-responses-latent-curve-modeling doi:10.4135/9781526431592
Not straight line for all 9 time points; but yes within the 3 week units PS we tested cubic and it didn t work
evaluate change over time using 2 inter-correlated latent traits (i.e., starting trait mean and trait changeability). presumes that changes in a repeated measure are likely to be influenced by both what the original starting value was and the tendency of the person to change in their responding starting mean is kept constant because it influences equally all subsequent responses. change value increases in accordance with increase in time units. Ideally, change is linear over a period of time, but, unsurprisingly, a constant linear increase might not be a good model for every event. Alternative estimators (e.g., quadratic, cubic) can be fitted to see if that helps the model correspond better with the data.
The GPA (marked in red) is regressed onto the intercept and slope factors to evaluate whether those values are caused by course entry GPA. the GPA is also regressed onto the Mid-Term Test score (marked in green) as performance on the test is probably a consequence of general academic ability. The green lines show the effect of emotion start values and change rates upon mid- term test scores.
Open AMOS LCM Chilled + gpa no paths.amw Link file to Chilled Week 3.sav Draw paths from Slope and Intercept to Week 7, 8, and 9 Add appropriate weight values Correlate repeated items Correlate slope & intercept Regress GPA onto slope & intercept
