P-Value Plots in Meta-Analysis Studies
Explore the significance of p-value plots in meta-analysis studies to evaluate claims based on research findings. Learn how to interpret randomness, effects, and p-hacking through visual representations of p-values.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
MetaEval: A JMP Add-In to evaluate a claim coming from a meta-analysis S. Stanley Young CGStat genetree@bellsouth.net Paul Fogel Consultant, Paris, France JMP Discovery Summit Cary, NC 10/23-26/2018 1
What is a p-value plot? Rank order the p-values from smallest to largest. Plot against the integers: 1, 2, 3, If you see a 45-degree line, then randomness. If you see a less than 45-degree line, then some effect. If you see bi-linear, then stop and think. 2
P-value plots (b) Simulated Random (c) Real Random (a) Simulated Real (d) Bi-linear ozone 3
What is a meta-analysis? 1. Pick a question 2. Computer search the literature 3. Select papers (selection bias) 4. Combine the individual results 5. Publish a paper Cookbook: A team can publish one paper per week. Google Scholar search for meta-analysis => 3.3M 4
Forest Plot PCB/Breast Cancer) CLs to p-values 1. Computations in MetaEval 2. SE = CLhigh - CLlow 3. Z = (RR-1.000)/SE 4. p = Normal (Z) 5. P-value plot: rank p-values plot against the integers See JMP Add-in Forest Plot by Richard Zink Claim: Our meta-analysis based on the selected studies found group II and group III PCB exposure might contribute to the risk of breast cancer. 5
P-value plot Our impression is that we are looking at randomness. 6
Lancet Study Interpretation In view of both the magnitude of the risk and the prevalence in the population, air pollution is an important trigger of myocardial infarction, Air pollution is a killer! 7
P-value plot Bi-linear: small p-values due to p-hacking and random p-values. 9
Six p-value plots from another paper. Note well, the essentially bi-linear nature of the plots is consistent with p-hacking on the left and random on the right. 10
Science Incentives 1. Need p-value less than 0.05 to publish. 2. Positive results are funded. 3. Negative result stops funding. 4. Confirmation bias. 5. Technically easy to get p-value less than 0.05. a. Manipulate data. b. Test lots of questions, p-hacking. c. Compute multiple models, p-hacking 11
Information Stan Young, genetree@bellsouth.net Paul Fogel, paul_fogel@hotmail.com JMP Add-in to go from RR, CLs to p-values is available from the author. Virtually all observational studies do not correct for multiple testing. P-value plots appear to be a sensitive way to check heterogeneity. This area is wide open. Young is willing to collaborate. 12
