Safety Analysis of Time-Varying Exposures in Post-Marketing Studies

Comparative safety analysis of time-varying exposures in post marketing observational studies Sean Yiu

Setting Ocrevus (OCR) is an approved treatment for adult patients with relapsing or primary progressive forms of multiple sclerosis (MS). As part of the license agreement with the FDA, Roche is required to conduct a prospective longitudinal observational study in adult patients with MS exposed to OCR. Objective: To determine the incidence and mortality rates of breast cancer and all malignancies. Minimum follow-up: All patients enrolled in the study should be followed for a minimum of 5 years or until death following their first exposure to OCR. Comparators: The protocol must specify two appropriate populations to which the observed incidence and mortality rates will be compared. 2

FDA requests during review of protocol Include an internal comparator consisting of patients with MS who receive medical care from the same clinical sites as the OCR-exposed patients. Collection of all potential confounding factors. Use of appropriate causal inference models. Please submit a detailed statistical analysis plan (SAP) for review. Include the following information in the SAP: 1. Plans for how to analyse study participants who switch MS drugs during the study period; 2. Plans for descriptive and comparative analyses, including how to control baseline and time-dependent confounders. 3

Estimand for comparative safety analysis Population: OCR-exposed patients Outcome variables: Time to onset of malignancies, death, serious infections, etc. Exposures: 1.Everexposed to OCR vs disease modifying therapy (DMT) comparator (comprises of 6 other DMTs) 2. Currentlyexposed to OCR vs DMT comparator Population-level summary: hazard ratio Intercurrent event (IE): Initiation of alternative DMTs to the one received at baseline IE handling strategy: hypothetical strategy where switching would not occur Causal representation of estimand (under proportional hazards assumption) For all time ? > 0, the estimand can be represented as log ??(???? ? =1,??? ? =0> ?|??? 0 = 1) log ??(???? ? =0,??? ? =1> ?|??? 0 = 1) 4

Estimating treatment effects for the target population: Dealing with baseline confounding Strategy: Use patients exposed to OCR at baseline to estimate ?? ???? ? =1,??? ? =0> ? ??? 0 = 1 Use patients exposed to DMT comparator at baseline to estimate ?? ???? ? =0,??? ? =1> ? ??? 0 = 1 Patients exposed to DMT comparator at baseline will not be exchangeable with those exposed to OCR at baseline. Propensity scores (PS) can be used to account for different likelihoods of being prescribed OCR versus other MS drugs. We proposed to use inverse probability of treatment weighting (IPTW) by reweighting patients exposed to DMT comparator at baseline by the odds of receiving OCR at baseline, i.e. PS/(1-PS), where PS=pr(OCR | baseline confounders). 5

Approaches to handle treatment switching (in the absence of time-dependent confounding) 6

Approach 1: Censor outcomes at time of IE (Censoring method) A typical approach to deal with initiation of alternative treatments for comparative effectiveness of randomized studies is to censor patients at the time of the IE. Exposure effects are then estimated by contrasting outcomes across exposure groups. OCR cohort DMT comparator cohort Discard data post IE Discard data post IE Only use data prior to IE Only use data prior to IE X X Initiate DMT comparator at baseline Discontinue DMT comparator Initiate OCR Initiate OCR at baseline Discontinue OCR Initiate DMT comparator Advantage:The censoring method should provide unbiased exposure effect estimates when time-dependent confounders are absent. Concern: Can lead to a large loss of data and highly imprecise inferences. 7

Approach 2: Rebaseline at time of IE (Rebaselining method) Define two non-mutually exclusive exposure groups based on whether patients are exposed to OCR/DMT comparator at baseline or on study. OCR exposed group DMT comparator exposed group Time zero Time zero Patient A X X Initiate DMT comparator at study baseline Initiate OCR at study baseline Time zero Time zero X X Patient B Initiate OCR on study Initiate OCR at study baseline Initiate DMT comparator at study baseline Discontinue DMT comparator Discontinue OCR Initiate DMT comparator Under this approach, patient B will contribute safety outcomes and/or follow-up time to both exposure groups. Main appeal: All data are used. 8

Limitations of the rebaselining method Limitation 1: Double counting of safety events and follow-up time can lead to overly precise inference. Limitation 2: The rebaselining approach can lead to attenuated exposure effect estimates and decreased power. Recall that under the rebaselining approach, patients that join a new exposure cohort during their follow-up will have been exposed to a study drug. Hence, previous exposure to study drug may now be a confounder. Initiate OCR at baseline Initiate DMT comparator during follow-up Initiate OCR at baseline Consequently, patients may not be exchangeable across the exposure groups from the rebaselining method. Second, patients across exposure groups are being compared at different times during the study. This is inconsistent with the definition of the estimand. 9

Approach 3: Time-varying Cox model FDA recommendation: Given the possibility of treatment switching, use a time-varying exposure Cox model with inverse probability treatment weight (IPTW) or matching adjusting for baseline covariates in the main analysis. Under the assumption that there is no interaction between the OCR and DMT exposure effects, the hazard function can be modelled as ? = 0(?) exp ???(?) + ???(?) ??? ? and ???(?) specify whether the patient was previously or is currently exposed to OCR or DMT comparator at time ?. exp( ) characterizes the change in risk from exposure to OCR, including for patients that are not exposed to DMT comparator, and similarly exp( ) provides an analogous characterization for the DMT comparator. Therefore, the target estimand can be estimated as exp( ). 10

Advantages of the Time-varying Cox model Critical assumption: Treatment switching does not modify the exposure effects. Advantage 1:Uses all data without double counting of events or follow-up time. Advantage 2:Adjustment for exposure to previous study drugs is provided by the model (helps to ensure exchangeability of patients across time). Advantage 3: Patients across exposure groups are being compared at the same time points during the study. Note that the time-varying Cox model will be essentially equivalent to the censoring method for the ever exposed analysis. X Initiate OCR at study baseline Discontinue OCR during the study Initiate DMT comparator during follow-up Safety events that occur after initiation of both drugs cannot be attributed to one drug more than the other in the current model. 11

Simulations summarizing differences in methods under random treatment switching for ever-exposed analysis Data generating mechanism 4000 new users of OCR, and 1333 new users of DMT comparator at study baseline. Planned follow-up duration is 90 months. Method Bias on log HR scale (average HR) Power Chance of switching from OCR to DMT comparator is 5% within first 6 months, and 5% from 6 months to study end. Censoring 0.012 (1.54) 72.7% Rebaselining (without cluster robust SE) -0.103 (1.37) 51.0% Chance of switchingfrom DMT comparatorto OCR is 20% by study end. Rebaselining (with cluster robust SE) 54.1% Chance of malignancy is 5% by study end. HR for OCR effect on risk of malignancy is 1.5. Time-varying Cox model 0.012 (1.54) 72.7% Chance of dropping out from study is 20%. Results Rebaselining method underestimates the true HR and has lower power. Use of cluster-robust standard errors increased power for the rebaselining method. Censoring and time-varying Cox model were essentially equivalent and had highest power. 12

Simulations summarizing differences in methods under random treatment switching for currently exposed analysis and Data generating mechanism 4000 new users of OCR, and 1333 new users of DMT comparator at study baseline. Method Bias on log HR scale (average HR) Power Planned follow-up duration is 90 months. Censoring 0.002 (1.52) 72.5% Chance of switching from OCR to the DMT comparator and vice versa is 20% by study end. Rebaselining (without cluster robust SE) -0.038 (1.46) 82.6% Chance of malignancy is 5% by study end. Rebaselining (with cluster robust SE) 87.7% HR for OCR effect on risk of malignancy is 1.5. Time-varying Cox model 0.001 (1.52) 83.3% Chance of dropping out from study is 20% Results Rebaselining method underestimates the true HR. Use of cluster-robust standard errors increased power for the rebaselining method. Censoring and time-varying Cox model were essentially equivalent, and time-varying Cox model had highest power. 13

Approaches to handle time-dependent confounding 14

Marginal structural Cox models for the covariate dependent treatment switching setting When time-dependent confounders of the treatment switching and outcome process are present, the time-varying Cox model is no longer guaranteed to provide unbiased inference for the target estimand. Time-varying disability status X Discontinue DMT comparator during follow-up Initiate DMT comparator at study baseline Initiate OCR during follow-up A natural solution to this problem is to simulate the hypotheticaltarget trial where patients do switch treatments at random (i.e. independent of time dependent confounders). In this setting, the time-varying Cox model can be extended to a marginal structural Cox model (MSCM). Structural => Time-varying Cox model is for potential outcomes from the target trial (not the observed outcomes). 15

Regression adjustment of time-dependent confounders may fail to estimate parameters of the MSCM Time-varying disability status X Discontinue DMT comparator during follow-up Initiate DMT comparator at study baseline Initiate OCR during follow-up Adjusting for time-varying disability would reduce confounding bias for the effect of future treatments on safety outcomes. However, adjusting for time-varying disability would block some of the effect of previous treatments on safety outcomes. Therefore, adjusting for time-varying confounders that are affected by previous treatments could lead to downward biased effect estimates for exposure to previous treatments. 16

Inverse probability weighted estimation of marginal structural models Possible solution: use IPW to emulate the target trial with an unconfounded treatment switching process. Involves discretizing the time scale, e.g. into months, and then assigning weights to each patient at each time point. Patients are assigned the following weights at month ?: ? ?? ??? ? ??? ? 1 ,??? ? 1 ,?) ?? ??? ? ??? ? 1 ,??? ? 1 ,?) ?? ??? ? ??? ? 1 ,??? ? 1 ,? ? ,?) ?? ??? ? ??? ? 1 ,??? ? 1 ,? ? ,?) ?=1 ?(?) Data generating mechanism for the pseudo population after weighting: X Initiate DMT comparator at study baseline Discontinue DMT comparator during follow-up Initiate OCR during follow-up More details of this and other solutions including g-computationformula and g-estimation of structural nested models can be found in Daniel et al (2013) and Chapter 21 of Hern n and Robins (2020). 17

FDA feedback on proposed MSCM FDA feedback: You proposed to use the marginal structural models (MSMs) in the main analyses. There are practical issues that are of concern. For example: 1. MSM has more stringent model specification requirements for the propensity score (PS) estimations over time. If the PS model is misspecified at one of the time points, then the effect estimate could be biased. 2. Pre-determined fixed time intervals are required for the MSMs. Variable values that are not captured at the pre- determined time points may require assumptions to carry values forward or backward. Use MSMs in the exploratory analyses only. The impact of time-varying confounders can be further explored by the MSMs in the exploratory analyses. In the main comparative analyses, use the IPTW to balance the baseline characteristics and fit a weighted time-dependent Cox regression model to study the association between exposures and outcomes, accounting for time-varying exposures and time-varying covariates. Response: Agreed to estimate parameters of the MSCM by adjusting for time-varying confounder that are not intermediates in the model for the hazard. 18

FDA feedback continued 1. FDA feedback: We acknowledge that you plan to calculate the inverse probability weights using the baseline covariates in the main analysis. Then, the weighted time-dependent Cox regression model will be used to account for time-varying exposures and time-varying confounders that are not intermediates. We request that you also include time-varying covariates in the weighted time-dependent Cox regression model, since those covariates could be risk factors that may impact the outcomes over time. Response: Agree to proposal. 2. FDA feedback: Justify whether EDSS and BMI are time-varying confounders or time-varying covariates. List any other potential time-varying covariates or time-varying confounders. Response: Provided a list of variables and classified them based on clinical understanding. In future, it might be more helpful to provide the assumed DAG. 3. FDA feedback: Plan to provide specifications and diagnostics of the MSM model, such as time intervals of evaluations and distributions of weights at all time points. Response: Monthlyvisit gaps are possible, and assessment of outlying weights and whether the weights have unit mean will be provided. 19

Summary Randomized controlled trials are often underpowered to detect treatment effects on important long-term safety events, e.g. malignancies. Therefore, long-term post approval safety studies are sometimes mandated, and these are often observational due to feasibility constraints. Additionally, inclusion of an internal comparator and comparative safety analysis may be an integral part of fulfilling post marketing requirements from health authorities. Causal inference methodology for observational data, e.g. to estimate time-varying exposure effects in the presence of time-dependent confounding, appears to be necessary (requested by the health authority), if not essential (to align estimators to estimands, clarify assumptions, and to highlight limitations). 20

References Daniel RM, Cousens SN, De Stavola BL, Kenward MG, Sterne JA. Methods for dealing with time-dependent confounding. Stat Med. 2013 Apr 30;32(9):1584-618. doi: 10.1002/sim.5686. Epub 2012 Dec 3. PMID: 23208861. Hern n MA, Robins JM (2020). Causal Inference: What If. Boca Raton: Chapman & Hall/CRC. 21

Doing now what patients need next