Advanced Methods and Analysis for Learning and Social Sciences - Meta-Analysis Insights

advanced methods and analysis for the learning n.w
1 / 41
Embed
Share

Explore the concept of meta-analysis, its uses, key challenges, and methods for combining significance and effect sizes in research. Discover when to apply weighting and the key assumption of independence between studies. Dive into the intricacies of linear correlation and converting correlation coefficients for statistical analysis.

  • Methods
  • Analysis
  • Meta-Analysis
  • Social Sciences
  • Learning
rayaanacharya
omaza Follow

Uploaded on | 0 Views

Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.

Download Presentation

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.

E N D

Presentation Transcript

  1. Advanced Methods and Analysis for the Learning and Social Sciences PSY505 Spring term, 2012 February 22, 2012

  2. Todays Class Meta-Analysis

  3. Meta-Analysis What is it?

  4. Meta-Analysis What is it usually used for?

  5. Meta-Analysis What are the key challenges?

  6. Meta-Analysis What are the key challenges? Lack of detail in reports (p<0.05) Inconsistent reports (r, p, d/ ) File-drawer problem Construct-name mapping

  7. Combining Significance Stouffer s Z Z sqrt(K)

  8. Combining Significance Stouffer s Z sum(Z) sqrt(sum(Var(Z))2) Var(Z)=1 because it s the normal distribution

  9. What if you have p?

  10. What if you have t?

  11. What if you have r?

  12. Asgn. 5 Problems 1-3

  13. Weighting When do R&R recommend weighting Z s? Is it a good idea?

  14. Key assumption Independence between studies When might this assumption be violated? If independence not met, there are other tests that can be used See chapter

  15. Combining Effect Size Case of linear correlation r

  16. Combining Effect Size Convert r to Fisher z (not the same as Z!) Using a table or function Why? As correlation approaches 1 or -1, the distribution of correlation becomes non-normal The 95% confidence interval for a correlation of 0.9 might include 1.1, but correlation can t be greater than 1 Fisher z adjusts this to make all distributions normal, making it possible to integrate across correlations

  17. Combining Effect Size Combine Fisher z sum(z) K

  18. Combining Effect Size Convert Fisher z back to r

  19. Key assumption Independence between studies When might this assumption be violated? If independence not met, there are other tests that can be used See chapter

  20. What about d/ Convert it to r Then conduct meta-analysis on r Different equations for this conversion depending on properties of the data set For more info, see p. 239 of Cooper, H., Hedges, L.V. (1994) The Handbook of Research Synthesis.

  21. Cool thing Same methods can be used to compare between significance values or correlations Subtract values rather than summing them

  22. Example What is the significance of each study? What is the significance of the two studies? What is the difference between the studies? Z=1.9, Z=2.2 Z=1.9, Z= -0.5

  23. Evaluating detector goodness Let s do problem 4 together

  24. Evaluating detector goodness How do we get an A from this data? (ignoring non-independence)

  25. Evaluating detector goodness How do we get SE(A ) from this data? (ignoring non-independence)

  26. Evaluating detector goodness How do we compare A to chance? (ignoring non-independence)

  27. Evaluating detector goodness Now, why was this the wrong thing to do? (ignoring non-independence)

  28. Evaluating detector goodness Re-doing the procedure accounting for independence Compute A for each student Compute Z for each student Use Stouffer s Z to integrate across students

  29. Evaluating detector goodness When is this method inappropriate/useless?

  30. Cleaning the Registers At this point, every student should have handed in three assignments If you haven t, come talk to me after class You now need to do 3 of Assignments 6-10

  31. Asgn. 6

  32. Next Class Monday, February 27 3pm-5pm AK232 Regression and Regressors Readings Ramsey, F.L., Schafer, D.W. (1997) The Statistical Sleuth: A Course in Methods of Data Analysis. Sections 7.2-7.4, 9.2- 9.3, 10.2-10.3 Witten, I.H., Frank, E. (2005)Data Mining: Practical Machine Learning Tools and Techniques. Sections 4.6, 6.5. Assignments Due: 6. Regression

  33. The End

  34. Bonus Slides If there s time

  35. BKT with Multiple Skills

  36. Conjunctive Model (Pardos et al., 2008) The probability a student can answer an item with skills A and B is P(CORR|A^B) = P(CORR|A) * P(CORR|B) But how should credit or blame be assigned to the various skills?

  37. Koedinger et al.s (2011) Conjunctive Model Equations for 2 skills

  38. Koedinger et al.s (2011) Conjunctive Model Generalized equations

  39. Koedinger et al.s (2011) Conjunctive Model Handles case where multiple skills apply to an item better than classical BKT

  40. Other BKT Extensions? Additional parameters? Additional states?

  41. Many others Compensatory Multiple Skills (Pardos et al., 2008) Clustered Skills (Ritter et al., 2009)

Related


No related presentations.

More Related Content