Addressing Missing Participant Data in Systematic Reviews
This content explores strategies for addressing missing participant data in systematic reviews, focusing on dichotomous outcomes in research studies. It delves into identifying and managing instances where participant data are absent, offering insights on implications and solutions. The discussion includes workshop plans, dealing with missing participant data at different levels, and considerations for data analysis in systematic reviews.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Addressing missing participant data in systematic reviews: Part I Dichotomous outcomes Elie Akl, Shanil Ebrahim, Bradley Johnston, Pablo Alonso, Matthias Briel, Gordon Guyatt
Disclosures No conflicts of interest to disclose This work was funded by the Cochrane Methods Innovation Fund
Workshop plan Missing participant data at the RCT level Missing participant data at the SR level Practical issues Discussion
Workshop plan Missing participant data at the RCT level Missing participant data at the SR level Practical issues Discussion
Missing Participant Data MPD refers to: participants excluded from the analysis of the effect estimate in the primary study because no data are available MDP does not refer to: Missing studies (e.g., unpublished studies); Missing outcome data (e.g., unreported outcomes); Missing summary data (e.g., unreported SD); Missing study-level characteristics (e.g., mean age, for subgroup or meta-regression analyses)
Dealing with MPD at the RCT level 87% of RCTs published in high impact medical journals had participants with missing data for the primary outcome The median percentage of participants with missing data was 6% (inter-quartile range 2% to 14%) Akl et al. BMJ. 2012 May 18;344:e2809
Primary outcome: overall success Karl et al. BJU. 2010.106:24-6
Dealing with MPD at the RCT level Complete case analysis Make assumptions about the outcomes of participants with missing data: None suffered the outcome All suffered the outcome Best case scenario Worst case scenario
Dealing with MPD at the RCT level However, these assumptions are not plausible More plausible assumptions: based on RIMPD/FU event incidence among those with MPD (not followed-up) relative to the event incidence among those followed up RIMPD/FU = 2: event incidence among those with MPD is double the event incidence among those followed up Akl et al. BMJ. 2012 May 18;344:e2809
Exercise 1 20000 1000 1000 20 980 10 990 (not followed-up) (followed-up) (not followed-up) (followed-up) 200 240 ? ? events events
Dealing with MPD at the RCT level What are the advantages and disadvantages of: Complete case analysis Assuming none suffered the outcome Assuming all suffered the outcome Assuming best case scenario Assuming worst case scenario RIMPD/FU approach
Workshop plan Missing participant data at the RCT level Missing participant data at the SR level Practical issues Discussion
Dealing with MPD at the SR level What are the issues that systematic review authors need to deal with in relation to MPD?
Dealing with MPD at the SR level We will discuss how systematic reviewer authors need to: Deal with MPD when producing the pooled effect estimate for the primary analysis Assess risk of bias associated with MPD and the extent to which introduces confidence in results (quality of evidence)
Trial level - participant entry Randomized Trial level - participant flow Mistakenly randomized Adherent and followed-up Lost to follow-up Nonadherent Trial level - data collection Not Not Collected Collected Collected collected collected Trial level - data analysis Appropriately excluded Included Included Excluded Excluded Excluded Systematic review level - data availability Not Missing Available Available Available available Systematic review level - data analysis According to trial analysis ITT, per protocol or as treated CCA, or make assumptions Exclude
Dealing with MPD at the SR level The Cochrane handbook encourages systematic reviewer authors to re-analyze a study s effect estimate by including all randomized participants The handbook, however, fails to provide detailed guidance on how such analyses should be conducted
Proposal to handle MPD For the primary analysis: exclude participants with missing data (complete case analysis) When the primary analysis suggests important effect, we suggest sensitivity meta-analyses making different assumptions about the outcome of participants with missing data, to test the robustness of the results (the risk of bias) Akl et al. PLoS One. 2013;8(2):e57132
Judging RoB dichotomous MPD Results robust to a worst case scenario missing data does not represent a risk of bias Results not robust to worst case scenario test progressively more extreme assumptions culminating in a "worst plausible case Important changes in results with such sensitivity analyses suggest serious RoB
Example 1 Meta-analysis assessing effects of probiotics for prevention clostridium difficile-associated diarrhea Johnston et al. Ann Intern Med. 2012 Dec 18;157(12):878-88
Example 1 Based on these findings Would you judge the risk of bias as: low or high? Would you rate down the confidence in effect estimates (quality of evidence)?
Example 2 Meta-analysis comparing oral direct factor Xa inhibitors to low-molecular-weight heparin for thromboprophylaxis in patients undergoing total hip or knee replacement The primary analysis: complete case analysis factor Xa inhibitors reduced the incidence of symptomatic deep venous OR 0.46 (0.30-0.70) Neumann et al. Ann Intern Med. 2012 May 15;156(10)
Example 2 Two sensitivity analyses based on extreme but plausible assumptions : RILTFU/FU 2 and 3 for the intervention arm and 1 for control arm. The effect estimates did not change appreciably: OR 0.54 (0.37-0.80), 0.59 (0.40-0.87) respectively Neumann et al. Ann Intern Med. 2012 May 15;156(10)
Example 2 The results would lose statistically significance, OR 0.84 (0.59-1.20), when we assumed: the lowest incidence among intervention arms of all included trials for those with missing data in the control group the highest incidence among control arms of all included trials for those with missing data in the intervention group Neumann et al. Ann Intern Med. 2012 May 15;156(10)
Example 2 Based on these findings Would you judge the risk of bias as: low or high? Would you rate down the confidence in effect estimates (quality of evidence)?
Workshop plan Missing participant data at the RCT level Missing participant data at the SR level Practical issues Discussion
Practical issues Identifying in the RCT report, which participants were actually followed up, and which participants having data missing Automatic integration of MPD in the analysis
Workshop plan Missing participant data at the RCT level Missing participant data at the SR level Practical issues Discussion
Discussion Is the proposed approach reasonable? Is the proposed approach feasible?
