Understanding the Importance of Effect Sizes in Meta-Analysis
Explore the significance of effect sizes in research through real-life scenarios to understand the impact of findings on decision-making. Learn how to interpret effect magnitudes for meaningful insights in various fields like medicine and social sciences.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
EVAL 6970: Meta-Analysis Review of Principles and Practice of Meta-Analysis Dr. Chris L. S. Coryn Spring 2011
Agenda Review of principles and practice of meta-analysis Questions
Why Effect Sizes? Imagine your doctor gave you the following information Research shows that people with your body-mass index and sedentary lifestyle score on average 2 points lower on a cardiac risk assessment test in comparison to active people with a healthy body weight Would this prompt you to make drastic changes to your lifestyle?
Why Effect Sizes? Now imagine your doctor said this to you instead Research shows that people with your body-mass index and sedentary lifestyle are four times as likely to suffer a serious heart attack within 10 years in comparison to people with a normal body weight Would this prompt you to make drastic changes to your lifestyle?
The Problem of Interpretation It is not sufficient to know the size and direction of an effect Effect magnitudes must be interpreted to extract meaning Effects by themselves are meaningless unless they can be contextualized against some frame of reference
The Problem of Interpretation Medicine is a special case when it comes to reporting results in metrics that are widely understood Most people have heard of cholesterol, blood pressure, the body-mass index, blood-sugar levels These metrics are easily amenable to interpretation
The Problem of Interpretation In the social sciences many phenomena can be observed only indirectly Self-esteem, trust, satisfaction, and depression are typically measured using scales and such scales are usually considered arbitrary when there is no obvious connection between a score and an individual s actual state or when it is not known how a one-unit change on the score reflects change in the underlying construct These metrics are useful for gauging effect sizes, but make interpretation difficult
Cohens Effect Size Benchmarks Effect Size Classes Test Effect Size Small Medium Large ?, ,Hedge s ? Comparison of two independent means .20 .50 .80 ? Comparison of two correlations .10 .30 .50 Cohen s ? Difference between proportions .05 .15 .25 ? .10 .30 .50 Correlation ?2 .01 .09 .25 ?2 ANOVA .01 .06 .14 ?2 Regression .02 .13 .26
Principles of Meta-Analysis Formulate statement of problem Identify and retrieve literature Code literature Analyze data Interpret results
Forest Plot (Fixed-Effect) Study name Odds ratio and 95% CI Mrozek-Budzyn et al. (2010) Takahashi et al. (2003) Fombonne et al. (2004) Uchiyama et al. (2007)b Uchiyama et al. (2007)a Smeeth et al. (2004) Aldridge-Sumner (2006) Taylor et al. (1999) Madsen et al. (2002) Uchiyama et al. (2007)c DeStefano et al. (2004)b DeStefano et al. (2004)a 0.1 0.2 0.5 1 2 5 10
Forest Plot (Random-Effects) Study name Odds ratio and 95% CI Mrozek-Budzyn et al. (2010) Takahashi et al. (2003) Fombonne et al. (2004) Uchiyama et al. (2007)b Uchiyama et al. (2007)a Smeeth et al. (2004) Aldridge-Sumner (2006) Taylor et al. (1999) Madsen et al. (2002) Uchiyama et al. (2007)c DeStefano et al. (2004)b DeStefano et al. (2004)a 0.1 0.2 0.5 1 2 5 10
Heterogeneity Statistics ?2 Effect Size and 95% CI Test of Null Heterogeneity ?2 ?2 ???2 ??2 ? ? ?? ?? ? ? ? ??(?) ? ? Model Fixed 12 0.94 0.85 1.02 -1.44 0.15 29.72 11 0.00 62.99 0.05 0.05 0.00 0.23 Random 12 0.87 0.72 1.06 -1.42 0.16
Funnel Plot (To Left of Mean) Funnel Plot of Precision by Log odds ratio 14 12 10 Precision (1/Std Err) 8 6 4 2 0 -2.0 -1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 2.0 Log odds ratio
Funnel Plot (To Right of Mean) Funnel Plot of Precision by Log odds ratio 14 12 10 Precision (1/Std Err) 8 6 4 2 0 -2.0 -1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 2.0 Log odds ratio
Publication Bias Statistics Duval and Tweedie s Trim and Fill = 3 (to right of mean) and 0 (to left of the mean) Kendall s ?-b = -0.439 (one-tailed ? = 0.023; two-tailed ? = 0.046) Egger s Test of the Intercept indicates an intercept of -1.269, with ? = 1.688, ?? = 10, and a two-tailed ?-value of 0.122 Orwin s Fail-Safe N = 11
