Practical Experience in Utilizing Real-World and Clinical Trial Data for Treatment Effects Assessment

Practical experience in utilizing real Practical experience in utilizing real- -world and clinical trial data for assessing treatment effects clinical trial data for assessing treatment effects world and L. J. Wei Harvard University

Two groups of patients (real world and clinical trial) Two groups of patients (real world and clinical trial) These two groups of patients can be quite different. What is the estimand (target population; endpoints; treatment contrast)? Are endpoint assessments similar between two groups? What is the index time (time zero) for patient in the real-world setting (via repeated biomarkers to define index time)? What are the baseline variables? How to select patients in the RW matching those in the trial (the target is the trial population or the real world population)? How to make further adjustments via analytic procedures?

Various methods for adjustments Matching baseline covariates (difficult for multiple variables) Propensity score matching (pros and cons) Inverse probability treatment weighting Outcome regression (not Analysis of covariance) Bayesian dynamic borrowing (cannot handle one arm trial) Entropy balancing (weighting individual patient by matching the moments) Double robustness (outcome regression and the propensity score)

In theory, it seems not a big deal for borrowing information from RW data, but in reality .. Example (single arm and RWD)

Circulation: Cardiovascular Quality and Outcomes (2019) Integrating Data From Randomized Controlled Trials and Observational Studies to Assess Survival in Rare Diseases Insights From Pulmonary Arterial Hypertension By A. Torbicki, MD et al.

SERAPHIN SERAPHIN A global, A global, multicenter multicenter RCT RCTwith with macitentan macitentanin PAH in PAH A global, double-blind, randomized, placebo-controlled trial Patients were randomized (1:1:1) to placebo, macitentan 3 mg, or macitentan 10 mg. The primary end point was the time from the initiation of treatment to the first occurrence of a composite end point of death, atrial septostomy, lung transplantation, initiation of treatment with intravenous or subcutaneous prostanoids, or worsening of pulmonary arterial hypertension. A total of 250 patients were randomly assigned to placebo, 250 to 3-mg, and 242 to the 10-mg. The hazard ratio for the 10-mg vs placebo was 0.55 (97.5% CI, 0.39 to 0.76; P<0.001).

. SERAPHIN: Primary & secondary endpoints All-cause mortality up to EOS Morbidity and mortality up to EOT 100 25 Patients without the event (%) Patients with the event (%) Placebo 80 20 Macitentan 10 mg 60 15 Placebo 40 10 Macitentan 10 mg 5 20 HR= 0.55 (p < 0.0001) HR=0.77 (p = 0.25) 0 0 0 6 12 18 24 30 36 0 6 12 18 24 30 36 Time from treatment start (months) Time from treatment start (months) Patients at risk Patients at risk 242 208 187 250 188 160 171 155 91 41 135 122 64 23 242 230 225 250 231 219 221 211 132 63 214 208 124 51 EOS: end of study; EOT: end of treatment Pulido T, et al. N Engl J Med 2013; 369: 809-18.

How about survival benefit? Some of the patients in the control arm were crossed over to the treated arms after experiencing non-fatal primary events. It is difficult to conduct a trial with respect to survival endpoint. Could we use external data to assess if the treatment had survival benefit?

REVEAL Registry Study for PAH REVEAL A US-based, multicenter registry of patients designed to provide information on the demographics, clinical course and management of 3515 patients with PAH

Challenges of using external controls REVEAL enrolled a broader population of PAH patients than SERAPHIN for disease classification, severity, and age et al. Need to select a cohort of REVEAL patients who could have been enrolled in SERAPHIN The final cohort selection variables were based on SERAPHIN inclusion and exclusion criteria.

Selection of REVEAL Cohort SERAPHIN RCT eligibility criteria 12 12 years years WHO WHO FC FC I I- -IV IV PAH PAH confirmed confirmed by by RHC RHC: : mPAP > 25 mmHg PCWP 15 mmHg PVR > 320 dyn sec cm-5 IPAH, HPAH or related to: CTD, simple corrected IPAH, HPAH or related to: CTD, simple corrected congenital shunts, HIV or drug/toxin exposure congenital shunts, HIV or drug/toxin exposure 6MWD 50 m 6MWD 50 m Body weight 40 kg Body weight 40 kg SBP 100 mg SBP 100 mg Estimated creatinine clearance 30 mL/min Estimated creatinine clearance 30 mL/min REVEAL N=3515 REVEAL Analysis Cohort (RAC) n=734

To further refine the comparability Building a survival prediction model using REVEAL data with the baseline covariates Initially, 29 covariates were considered from REVEAL;11 not available in SERAPHIN;18 covariates were used for modelling building. Predicting survival for each SERAPHIN patient using this model Getting the average of those 742 survival curves Comparing this average predicted survival with the survival observed in the SERAPHIN trial

Application of the prediction model to patients enrolled in SERAPHIN SERAPHIN All patients N=742 Predicted survival Application of survival prediction model SERAPHIN Placebo n=250 Observed survival SERAPHIN All patients N=742* SERAPHIN 10 mg macitentan n=242 Observed survival In the SERAPHIN study patients were randomised to placebo (n=250), macitentan 3 mg (n=250) and macitentan 10 mg (n=242).

Time to all-cause death up to 3 years Placebo vs. predicted Macitentan 10 mg vs. predicted 30 30 HR: 0.65 (95% CI: 0.46 0.90); p = 0.010 20 20 Mortality (%) Mortality (%) 10 10 0 0 1 3 2 1 3 0 2 0 Year Year Patients at risk: Patients at risk: 250 219 208 51 225 211 63 242 Macitentan 10 mg (n=242) Predicted (n=742) Placebo (n=250)

In a setting of a rare disease where evaluating mortality is extremely challenging, using real-world and RCT data can bring valuable insights, The risk of mortality was underestimated in the observed SERAPHIN placebo arm due to cross-over to active therapy. The risk of mortality observed with macitentan 10 mg in SERAPHIN was 35% lower than the predicted mortality for all patients in SERAPHIN. Outcome regression method is intuitive and flexible for choosing the target population (not for PS or IPTW). If there were data from pure placebo in the trial, we may check the model fitting.

Conclusions Outcome regression seems intuitive and interpretable It provides survival curve estimate directly. If we have data from the control in the clinical study, we can then check the adequacy of the model. Need pre-specified analysis plan Designing a trial using RWD

Beyond methodology issues Real-World Evidence in Support of Oncology Product Registration: A Systematic Review of New Drug Application and Biologics License Application Approvals from 2015-2020 (Clinical care, 2022)

Early engagement is crucial to confirm appropriate RWD source and whether the RWE study should be designed as a natural history study for contextualization or as a historical/external control study for comparison. A pre-specified study protocol for transparency. FDA emphasized the selection of RWE population to be comparable to the pivotal trial population, by matching on trial inclusion and exclusion criteria to the extent possible and adjusting for the remaining imbalance in baseline characteristics with propensity score weighting methodology, such as inverse probability treatment weighting (IPTW). Regarding ORR, FDA had major critiques on real-world tumor assessment vs. Response Evaluation Criteria in Solid Tumors (RECIST) criteria with respect to the difference in frequency of scans and the baseline scan used to define response. Concordance analysis between real-world tumor response and trial response is encouraged. Lastly, methods to minimize confounding, including appropriate index date and reduction in missing values, need to be described. The impact of unmeasured confounding should be evaluated through quantitative bias analysis. D

Utilizing multiple outcomes from Utilizing multiple outcomes from comparative studies for assessing totality comparative studies for assessing totality of evidence on treatment efficacy/safety of evidence on treatment efficacy/safety 19

Example: Clinical studies for treating DMD Example: Clinical studies for treating DMD Duchenne muscular dystrophy (DMD) is a genetic disorder characterized by progressive muscle degeneration and weakness. Two independent comparative studies were conducted. Primary endpoint is the change in 6-minute walking distance (6MWD) from baseline to week 48. Secondary endpoints are changes in 10-meter walk/run, 4-stair climb and 4-stair descend times. 20

How to analyze data from multiple outcomes How to analyze data from multiple outcomes conventionally? conventionally? For each endpoint, use a single statistic such as the average difference between two arms to estimate the treatment effect. Obtain p-value and confidence interval. Apply a multi-stage testing procedure to claim statistical significance for the secondary endpoint. 21

Quantifying totality of treatment effect across multiple outcomes from independent comparative studies Study 007 Study 020 FavorsPlaceboFavors Rx FavorsPlaceboFavors Rx 6MWD (m) 6MWD (m) 6MWD -20 0 20 40 60 -20 0 20 40 60 6MWD 10-meterwalk/run 4-stairclimb 4-stairdescend 10-meterwalk/ run 4-stairclimb 4-stairdescend -20246 Change (second) -20246 Change (second) p=0.0086

23

JAMA OPEN Assessment of Treatment Effect With Multiple Outcomes in 2 Clinical Trials of Patients With Duchenne Muscular Dystrophy 24