Publication Bias in Public Health Emergencies

WHO Training Manual Ethics in epidemics, emergencies and disasters: Research, surveillance and patient care Learning Objective 5.2 Explain what is meant by publication bias and how it might affect the response to emergencies

Outline 1. Introduction 2. Reading 3. Tamiflu video 4. Case study and discussion 56-85 (30 min) Suggested time 0-20 (20 min) 21-40 (20 min) 41-55 (15 min) 86-90 (5 min) Activity Introduction Reading Video and group discussion Case study and discussion Summary and conclusion L.O. 5.2

Background Decisions about how to prepare for and respond to public health emergencies are likely to be informed by evidence generated from research. What is published and therefore what makes up the evidence base upon which decisions are informed has the potential to be biased. L.O. 5.2

Background Bias means [a]ny process at any stage of inference which tends to produce results or conclusions that differ systematically from the truth (Sackett, 1979, pg. 60). Publication bias refers to what occurs when what is published is systematically unrepresentative of what is available to be published (Rothstein et al., 2005). L.O. 5.2

Background The effect of publication bias could be that published studies do not accurately represent all data (in the case of publishing results from a single study) or all studies (in the case of publishing a systematic review or meta-analysis). Ultimately, the consequence of publication bias is the distortion of the evidence base (Eggar et al., 1995). If decisions are to be informed based on this evidence, those decisions may ultimately be based on incomplete, suppressed, or inaccurate information. L.O. 5.2

Forms of Publication Bias Positive results bias Positive results bias occurs when statistically significant positive results from a study are more likely to be published than results that support the null hypothesis or results that are inconclusive (i.e. negative results) (Hopewell et al., 2009). Hot stuff bias / lack of interest bias This bias occurs when a topic is popular (or not) within the scientific community or in the broader public domain, leading to an increased (or decreased) interest for publication on that topic L.O. 5.2

Forms of Publication Bias Confirmation bias Can exist when the results of research tend to match or support the interests, expectations, or hypothesis of the researcher or study sponsor (Mahoney, 1977). L.O. 5.2

Case Study Watch New doubts over Tamiflu : http://www.channel4.com/news/articles/science_techn ology/new+doubts+over+tamiflu/3454737.html L.O. 5.2

Case Study In 2003, a paper was published reporting the results of a study sponsored by F. Hoffmann-La Roche Ltd. about the impact of oseltamivir (brand name: Tamiflu) treatment on influenza-related lower respiratory tract complications (LRTCs) and hospitalizations. The paper reported that oseltamivir treatment of influenza reduces LRTCs, antibiotic use, and hospitalizations both for at- risk and healthy adults (Kaiser et al., 2003). L.O. 5.2

Case Study This study involved the analysis of 10 separate phase three randomized control trials (RCTs) sponsored by Roche, of which only 2 have been published in peer- reviewed journals. In a subsequent Cochrane review it was claimed that there is insufficient evidence to answer the question about whether oseltamivir is effective in reducing LRTCs, antibiotic use, or hospitalization without including the data from the 8 unpublished studies included in the initial paper (Jefferson et al., 2009). L.O. 5.2

Case Study Irrespective of this, the original study s evidence has been used by public health decision-makers to justify recommending oseltamivir as a treatment option in combating influenza, including pandemic strains of influenza (Godlee and Clarke, 2009). As a result, this has also led to the stockpiling of oseltamivir for use during an influenza pandemic. The authors of the Cochrane review conclude, [i]t is possible that there is a publication bias, especially as we know of eight trials that are unpublished and inaccessible Its direction might be in favour of exaggerating the treatment effect (Jefferson et al. 2009, pg. 6). L.O. 5.2

Case Study Questions 1. Could this case study indicate a publication bias? Why or why not? What kind of publication bias might exist in this case? 2. What could be done to prevent or mitigate the potential publication bias in this case? 3. How could this case affect the response to a public health emergency? L.O. 5.2

Discussion questions What is it that fundamentally drives publication? Is this motivation any different during a public health emergency? If there is questionable evidence of effectiveness for a public health intervention (for example, the use of quarantine in response to an infectious disease like severe acute respiratory syndrome), what does that mean in terms of the justifiability of using that public health intervention? See Bensimon and Upshur, 2007 for discussion on this topic. L.O. 5.2

Discussion questions Could a publication bias occur for the publication of papers on the ethics of public health emergency preparedness and response? Could papers that come to conclusive answers about difficult ethical questions get published more easily or quickly? If the trustworthiness of the evidence base upon which decisions are informed has been diminished due to publication bias, how should decisions be made in light of this? Should peer-reviewers or journal editors (or some other body) have the ability to review study data themselves? L.O. 5.2

Summary What is published (therefore what makes up evidence upon which decisions are informed) may be biased Increased appetite for studies to inform planning and response efforts during public health emergencies May affect quality, nature and even findings of studies, affecting what is published Interests of all stakeholders must be examined to try to mitigate bias Data should also be publically available and evaluated There is a responsibility to work in solidarity in a public health emergency L.O. 5.2

Sources Bensimon CM, Upshur REG. Evidence and effectiveness in decisionmaking for quarantine. American Journal of Public Health, 2007, 97:S44 S48. Egger M, Smith GD. Misleading meta-analysis. British Medical Journal, 1995;310: 752-4. Godlee F, Clarke M. Why don t we have all the evidence on oseltamivir? British Medical Journal, 2009; 339: b5351. Hopewell S, Loudon K, Clarke MJ, Oxman AD, Dickersin. Publication bias in clinical trials due to statistical significance or direction of trial results (review). The Cochrane Library, 2009;1: 1-26. Jefferson T, Jones M, Doshi P, Del Mar C. Neuraminidase inhibitors for preventing and treating influenza in healthy adults: systematic review and metaanalysis. British Medical Journal, 2009; 339: b5106. Kaiser L, Wat C, Mills T, Mahoney P, Ward P, Hayden F. Impact of oseltamivir treatment on influenza- related lower respiratory tract complications and hospitalizations. Archives of Internal Medicine, 2003; 163: 1667-1672. Mahoney MJ. Publication prejudices: an experimental study of confirmatory bias in the peer review system. Cognitive Therapy and Research, 1977;1(2): 161-75. Rothstein HR, Sutton AJ, Borenstein M. Publication bias in meta-analysis. In Publication Bias in Meta- Analysis: Prevention, Assessment and Adjustments, eds Rothstein HR, Sutton AJ, Borenstein M. John Wiley & Sons, Ltd: Chichester, UK. 2006. Sackett DL. Bias in Analytic Research. Journal of Chronic Diseases, 1979;32: 51-63. L.O. 5.2

Acknowledgements Chapter author Smith, Maxwell, Dalla Lana School of Public Health and Joint Centre for Bioethics, University of Toronto, Toronto, Ontario, Canada L.O. 5.2