Comparative Effectiveness Research and Decision-Making in Healthcare

Putting it all together CIMPOD 2017, February 27-28 Miguel Hern n DEPARTMENTS OF EPIDEMIOLOGY AND BIOSTATISTICS

Putting it all together? In less than 30 minutes? Really? Estimating treatment effects (Drukker) Observational data (Rosen-Zvi) Propensity scores (Seeger) IV estimation (Swanson) IP weighting (Patel, Cain) G-formula (Young) Doubly-robust estimators (Rosenblum) Mediation analysis (Daniel) Machine learning (Rose) Hern n - Putting it all together 2

What is comparative effectiveness research? The generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat, and monitor a clinical condition or to improve the delivery of care Institute of Medicine 2009 Comparative effectiveness or comparative safety Patient-centered outcomes research Hern n - Putting it all together 3

The goal of comparative effectiveness research is to help decision-makers decide Patients, clinicians, policy makers, health care managers To assist [decision makers] to make informed decisions that will improve health care at both the individual and population levels Institute of Medicine 2009 Hern n - Putting it all together 4

Comparative effectiveness research and its purpose The generation and synthesis of evidence that compares the benefits and harms to make informed decisions that will improve health So CER is about estimating the effects of treatments where treatment means any health-related intervention Hern n - Putting it all together 5

CIMPOD 2017 Putting it all together Estimating treatment effects (Drukker) Observational data (Rosen-Zvi) Propensity scores (Seeger) IV estimation (Swanson) IP weighting (Patel, Cain) G-formula (Young) Doubly-robust estimators (Rosenblum) Mediation analysis (Daniel) Machine learning (Rose) Hern n - Putting it all together 6

The situation: We need to make decisions NOW Treat with A or with B? Treat now or later? When to switch to C? A relevant randomized trial would, in principle, answer each comparative effectiveness and safety question Interference/scaling up issues aside Hern n - Putting it all together 7

But we rarely have randomized trials expensive, untimely, unethical, impractical And deferring decisions is not an option no decision is a decision: Keep status quo Question: What do we do? Hern n - Putting it all together 8

Answer: We analyze observational data (pre-existing or collected for research) Epidemiologic studies Electronic medical records Administrative claims databases National registers Disease registries Other Hern n - Putting it all together 9

We analyze observational data but only because we cannot conduct a randomized trial Observational studies are not our preferred choice For each observational study with a causal goal, we can imagine a hypothetical randomized trial that we would prefer to conduct If only it were possible Hern n - Putting it all together 10

The Target Trial An analysis of observational data can be viewed as an attempt to emulate a (hypothetical) pragmatic randomized trial As suggested more or less explicitly by many authors, including Cochran, Rubin, Feinstein, Dawid, Robins Hern n, Robins. Am J Epidemiol 2016 Hern n - Putting it all together 11

Procedure to answer comparative effectiveness research questions Step #1 Describe the protocol of the target trial Step #2 Option A Conduct the target trial Option B Use observational data to explicitly emulate the target trial Hern n - Putting it all together 12

Key elements of target trial protocol Observational study needs to emulate Eligibility criteria Strategies Randomized assignment Start/End follow-up Outcomes Causal contrast(s) of interest Analysis plan Eligibility criteria Strategies Randomized assignment Start/End follow-up Outcomes Causal contrast(s) of interest Analysis plan Hern n - Putting it all together 13

CIMPOD 2017: Examples of Target Trials emulated using observational data Seeger Patel Swanson Young Cain Statin initiation and acute myocardial infarction Antiretroviral therapy and survival of HIV-positive children In utero exposure to atazanavir and neurodevelopment Sigmoidoscopy screening and colorectal cancer Antidepressants during pregnancy and depression Fish consumption and coronary heart disease Strategies for initiation of antiretroviral therapy and mortality Strategies to monitor HIV-positive patients and mortality Hern n - Putting it all together 14

CIMPOD 2017 Putting it all together Estimating treatment effects (Drukker) Observational data (Rosen-Zvi) Propensity scores (Seeger) IV estimation (Swanson) IP weighting (Patel, Cain) G-formula (Young) Doubly-robust estimators (Rosenblum) Mediation analysis (Daniel) Machine learning (Rose) Hern n - Putting it all together 15

Procedure to answer CER questions Step #1 Describe the protocol of the target trial Step #2 Option A Conduct the target trial Option B Use observational data to explicitly emulate the target trial Apply appropriate causal inference analytics to estimate the effects of interest Hern n - Putting it all together 16

Key elements of target trial protocol Observational study needs to emulate Eligibility criteria Strategies Randomized assignment Start/End follow-up Outcomes Causal contrast(s) of interest Analysis plan Eligibility criteria Strategies Randomized assignment Start/End follow-up Outcomes Causal contrast(s) of interest Analysis plan Hern n - Putting it all together 17

Observational analyses are, by definition, based on nonrandomized treatments Individuals receiving each treatment may not be comparable on average Confounding Emulation of randomization requires measuring and adjusting for confounding factors If insufficient information on baseline confounders then successful emulation of the target trial s random assignment is not possible Confounding bias Hern n - Putting it all together 18

Emulation of randomization requires adjustment for confounding factors Adjustment methods include matching, stratification/regression with or without propensity scores standardization/g-formula IP weighting Confounding adjustment methods are used to emulate randomization Sometimes referred to as causal inference methods Hern n - Putting it all together 19

CIMPOD 2017 Putting it all together Estimating treatment effects (Drukker) Observational data (Rosen-Zvi) Propensity scores (Seeger) IV estimation (Swanson) IP weighting (Patel, Cain) G-formula (Young) Doubly-robust estimators (Rosenblum) Mediation analysis (Daniel) Machine learning (Rose) Hern n - Putting it all together 20

Key elements of target trial protocol Observational study needs to emulate Eligibility criteria Strategies Randomized assignment Start/End follow-up Outcomes Causal contrast(s) of interest Analysis plan Eligibility criteria Strategies Randomized assignment Start/End follow-up Outcomes Causal contrast(s) of interest Analysis plan Hern n - Putting it all together 21

Emulation of randomization requires measuring and adjusting for confounding factors If insufficient information on baseline confounders then successful emulation of the target trial s random assignment is not possible Confounding bias Not quite true Hern n - Putting it all together 22

Two approaches to emulate randomization (Two approaches to adjust for confounding) 1. Correctly measure and appropriately adjust for all confounders Stratification/regression, matching with or without propensity scores G-methods: standardization/g-formula, g-estimation, IP weighting 2. Exploit sources of randomness in the data to adjust for confounding without measuring the confounders instrumental variable estimation, regression discontinuity, etc. Hern n - Putting it all together 23

Instrumental variable methods Do not require that all confounders are measured and adjusted for In fact, one does not need to even know which the confounders are But require other (often very strong) conditions for the validity of the estimates Hern n - Putting it all together 24

CIMPOD 2017 Putting it all together Estimating treatment effects (Drukker) Observational data (Rosen-Zvi) Propensity scores (Seeger) IV estimation (Swanson) IP weighting (Patel, Cain) G-formula (Young) Doubly-robust estimators (Rosenblum) Mediation analysis (Daniel) Machine learning (Rose) Hern n - Putting it all together 25

Key elements of target trial protocol Observational study needs to emulate Eligibility criteria Strategies assigned at start of follow-up Randomized assignment Start/End follow-up Outcomes Causal contrast(s) of interest Analysis plan Eligibility criteria Strategies followed from start of follow-up Randomized assignment Start/End follow-up Outcomes Causal contrast(s) of interest Analysis plan Hern n - Putting it all together 26

Classification of treatment strategies according to their time course Point interventions Intervention occurs at a single time Examples: one-dose vaccination, short-lived traumatic event, surgery Intention-to-treat effects in RCTs are about point interventions Sustained strategies Interventions occur at several times Examples: medical treatments, lifestyle, environmental exposures Many (most?) questions are about sustained exposures Hern n - Putting it all together 27

Classification of sustained treatment strategies Static a fixed strategy for everyone Example: treat with 150mg of daily aspirin during 5 years Dynamic a strategy that assigns different values to different individuals as a function of their evolving characteristics Example: start aspirin treatment if coronary heart disease, stop if stroke Hern n - Putting it all together 28

Choice of confounding adjustment method depends on type of strategies Comparison of strategies involving point interventions only All methods work if all confounders are measured or the instrumental variable conditions hold Comparison of sustained strategies Generally only g-methods work Developed by Robins and collaborators since 1986 Hern n - Putting it all together 29

Comparative effects of point interventions Time-fixed treatment implies time-fixed (i.e., baseline) confounding Any adjustment method will correctly adjust for measured baseline confounders e.g., outcome regression such as logistic or Cox regression Hern n - Putting it all together 30

Comparative effect of sustained strategies Time-varying treatments imply time-varying confounders possible treatment-confounder feedback Conventional methods may introduce bias even when sufficient data are available on Time-varying treatments and time-varying confounders G-methods can appropriately handle treatment- confounder feedback Sometimes referred to as causal methods Hern n - Putting it all together 31

Treatment-confounder feedback At: Antiretroviral therapy Y: Outcome Lt: CD4 cell count U: Immunologic status A1 A0 L1 Y U There is treatment-confounder feedback if the time-varying confounders are affected by previous treatment Confounder on the causal pathway NOT necessary for bias Hern n - Putting it all together 32

G-methods Parametric g-formula Robins 1986 G-estimation of nested structural models Robins 1989, 1991 IP weighting of marginal structural models Robins 1998 Doubly-robust versions Bang, Robins, Vanderlaan, Rotnitzky e.g., collaborative targeted maximum likelihood estimation Hern n - Putting it all together 33

CIMPOD 2017: Examples of sustained treatment strategies Patel Antiretroviral therapy and survival of HIV-positive children In utero exposure to atazanavir and neurodevelopment Young Fish consumption and coronary heart disease Strategies for initiation of antiretroviral therapy and mortality Cain Strategies to monitor HIV-positive patients and mortality Hern n - Putting it all together 34

CIMPOD 2017 Putting it all together Estimating treatment effects (Drukker) Observational data (Rosen-Zvi) Propensity scores (Seeger) IV estimation (Swanson) IP weighting (Patel, Cain) G-formula (Young) Doubly-robust estimators (Rosenblum) Mediation analysis (Daniel) Machine learning (Rose) Hern n - Putting it all together 35

Mediation analysis A particular application of causal inference Goal is estimating what component of the effect of treatment is, or is not, mediated by another factor Direct effect, indirect effect The target trial has two treatments The treatment of interest The mediator Need to emulate randomization for both the treatment and the mediator Stronger assumptions than non-mediational analyses Hern n - Putting it all together 36

CIMPOD 2017 Putting it all together Estimating treatment effects (Drukker) Observational data (Rosen-Zvi) Propensity scores (Seeger) IV estimation (Swanson) IP weighting (Patel, Cain) G-formula (Young) Doubly-robust estimators (Rosenblum) Mediation analysis (Daniel) Machine learning (Rose) Hern n - Putting it all together 37

Machine learning is a method for prediction Not for causal inference Lots of active research on incorporating machine learning into causal inference Hern n - Putting it all together 38

CIMPOD 2017 Putting it all together Estimating treatment effects (Drukker) Observational data (Rosen-Zvi) Propensity scores (Seeger) IV estimation (Swanson) IP weighting (Patel, Cain) G-formula (Young) Doubly-robust estimators (Rosenblum) Mediation analysis (Daniel) Machine learning (Rose) Hern n - Putting it all together 39

Remember Observational analyses are what we do when we cannot conduct a randomized trial In the absence of practical and ethical constraints, sane people will always prefer a randomized trial No alternative to observational studies So we better keep improving them because people will keep using (Big and Small) observational data to guide their decisions Hern n - Putting it all together 40