Enhanced Regimen Evaluation in Rare Disease Setting with Historical Data Borrowing

Adaptive Conditional Borrowing of Historical Data in a Randomized Control Trial in the Rare Disease Setting Peng Sun* Biogen October 11, 2019 *Joint work with Yiwei Zhang, Peitao Wu, Yingying Liu, John Zhong, Richard Foster and Charlie Cao 1

Outline Background information Elements of the proposed design with historical borrowing component Outline of the historical data borrowing approach Preliminary simulation results Conclusion 2

Introduction An efficacious disease modifying drug (D) for the treatment of a devastating rare disease has already been approved Despite strong efficacy of D, there is room for further efficacy improvement Primary objective of the proposed study: To evaluate the clinical efficacy of an enhanced regimen of D: Comparing to the current regimen, the enhanced regimen is targeted to double the steady state CSF concentration Historical data on the current regimen: N=73 patients treated with the current regimen from a completed Phase 3 study for the approval of D 3

Design of the Proposed Study 4

Primary Efficacy Endpoint and Type I Error Considerations Month 6 change from baseline in a functional endpoint (F) comparing the enhanced regimen with the current regimen Based on historical data, the Month 6 mortality rate on the current regimen is expected to be 14% Approaches considered for accounting for mortality: The trimmed mean approach The joint rank test The trimmed mean approach was selected based on Type I and power evaluations via a simulation study Type I error: Proposed one-sided alpha of 0.1 The current regimen is already highly efficacious The anticipated additional improvement for the enhanced regimen, although clinical meaningful, is not expected to be large (4.5 points increase in function and 3% improvement in Month 6 survival rate) Enrolment is expected to be very challenging 5

Illustration of the Trimmed Mean Approach Permutt and Li (2017) Hypothetical example: 1:1 randomization with 100 patients randomized to the treated arm and 100 to the control arm Treatment arm: 30 subject died; the remaining 70 subjects had change from baseline in functional score at the last scheduled visit Control arm: 50 subject died; the remaining 50 subjects had change from baseline in functional score at the last scheduled visit The trimmed mean approach compares the mean of change from baseline in functional score between the following two groups: From the control arm: All 50 surviving subjects From the treated arm: the 50 subjects (out of the 70 surviving subjects) who had the highest change from baseline in functional score p-value can be obtained from the reference distribution obtained via the re- randomization procedure 6

Illustration of the Joint Rank Test Berry et. al. (2013) Subject ranking: Subject who died ranks lower than subjects alive with Month 6 functional assessment Two subject who died: The subject who died at longer time ranks higher Two subjects alive with Month 6 functional score: The subject with higher Month 6 change from baseline in total functional score ranks higher CAFS score: Take a hypothetical subject A for example: Subject A will be compared with all other subjects in both treatment arms. For each comparison: Score of 1 if A ranks higher, score of -1 if A ranks lower, and score of 0 if it is a tie CAFS score for subject A: Sum of scores across all comparisons Joint rank test: Statistical test comparing the CAFS score ANCOVA model on CAFS score or generalized Gehan-Wilcoxon (GGW) test 7

Type I Error and Power Assessment 8 The trimmed mean approach was proposed based on the Type I error and power assessment

Outline of Historical Data Borrowing Approach [1] Objective: To borrow at least 25 subjects from historical data using a matching approach such as propensity score matching (PSM, Rosenbaum and Rubin [1985]) Notations: Treatment Arm (TA): Subjects randomized to the enhanced regimen Current Control (CC): Subjects randomized to the current regimen in the proposed study Historical Control Pool (HCP): N=73 patients treated with the current regimen from a completed Phase 3 study for the approval of D Historical Control (HC): Subset of HCP that is borrowed through the matching to TA Control Arm (CA): CC+HC 9

Outline of Historical Data Borrowing Approach [2] 10

Outline of Historical Data Borrowing Approach [3] Conditional borrowing: Criteria for acceptance of HC will be pre-specified in the following manner: For selected key covariates, the absolute standardized differences of means between HC and TA are all less than a pre-specified threshold (e.g., 0.25, Stuart [2010]), AND The observed Month 6 survival rate for CC falls within a pre-specified bound of the observed Month 6 survival rate for HC (e.g., 5-10% or a bound determined by standard error), AND The observed Month 6 mean change in functional score for CC falls within a pre- specified bound of the observed Month 6 change from baseline in functional score for HC (e.g., 1 1.5 points or a bound determined by standard error) Determination of conditional borrowing threshold will be based on evaluation of operating characteristics (e.g., borrowing rate, Type I error control, and power gains) Of note: There is no comparison of efficacy with TA when evaluating the feasibility of borrowing 11

Outline of Historical Data Borrowing Approach [4] Interim analysis to assess feasibility of borrowing: Timing: When N=75 subjects have been enrolled and at least N=36 with the opportunity for Month 6 assessment Goal: To evaluate the feasibility of borrowing from HCP. If deemed feasible, then no additional subjects will be enrolled. Otherwise, enrollment will continue until N=102 subjects have been enrolled (N=68 randomized to the enhanced regimen and N=34 to the current regimen) N=102 was selected so that the trimmed mean approach maintains at least 80% power assuming 4.5 points improvement in function and 3% improvement in survival Key challenge: At the interim, about half of the N=75 subjects don t yet have the opportunity for Month 6 assessment As a result, the interim decision should be based on the probability of being able to borrow at least N=25 subjects from HCP at the final Worst case scenario: At interim, it is decided not to increase the sample size and at final it is not feasible to borrow 12

Outline of Historical Data Borrowing Approach [4] Proposed decision framework at the interim: Use either caliper or optimal matching to bring in N=50 subjects in HC. Evaluate the probability of acceptance at final. If it is the pre-specified probability threshold, then stop further enrollment. Otherwise, use either caliper matching or optimal matching to bring in N=25 subjects in HC. Evaluate the probability of acceptance at final. If it is the pre-specified probability threshold, then stop further enrollment. Otherwise, continue enrollment until N=102 subjects have been enrolled Matching procedure at the final Regardless of interim decision to continue enrollment or not, the matching procedure at the final remains the same Use either caliper or optimal matching to bring in N=50 subjects in HC. Evaluate whether the pre-specified conditional borrowing criteria are met. If so, do final analyses comparing TA to CA (CC+HC) Otherwise, use either caliper or optimal matching to bring in N=25 subjects in HC. Evaluate whether the pre-specified conditional borrowing criteria are met. If so, do final analyses comparing TA to CA (CC+HC). Else, do final analyses comparing TA to CC. 13

Preliminary Simulation Results Limitations Goal: To evaluate borrowing rate, Type I error control, and power gain using simulation Some limitations of the current simulation work: The following step has not been built in: Matching N=25 HC when matching N=50 is not feasible The matching procedure is not repeated at the final. In other words, the HC identified at the interim is carried over to the final Criteria for balance in key covariates have yet to be implemented Matching method in the current simulation work: Propensity score matching with Caliper equal to 0.25 standard deviation of the logit of propensity scores (Rosenbaum and Rubin [1985]). Exploration of other matching methods is ongoing. The method for estimating acceptance probability is subject to further refinement 14

Preliminary Simulation Results Criteria for Efficacy Comparability between CC and HC 15

Preliminary Simulation Results Data Simulation Model 16 Note: For simulations, disease duration is obtained by bootstrapping from the completed trials

Preliminary Simulation Results A Note on Estimating Probability of Acceptance at the Time of Interim This involves predicting functional score and survival for patients in CC who do not yet have the opportunity for Month 6 assessment (N=13) Prediction of functional endpoint: Using linear mixed effect model Estimating probability of acceptance: Assume number of death of ranges from 0 to 4 Assuming 2 death as an example, there are 78 possible ways of assigning 2 death to 13 patients. We can estimate the proportion of cases (out of 78) of successful final borrowing For each scenario (number of death 0, 1, 2, 3, or 4), estimate the proportion of successful final borrowing Calculate the weighted proportion (p) using pmf of a binomial(13, 0.14) distribution Conclude feasible to borrow at the time of interim if p >70% Further work is ongoing to refine the method for estimating probability of acceptance 17

Preliminary Simulation Results Borrowing Rate 18

Preliminary Simulation Results Bounded Type I Error as Common Population Mean of TA and CC Shift Away from that of HCP 19 L1 = L2 = 1 was selected as the type I error is bounded by 15%

Preliminary Simulation Results Power Again Comparing TA to CC, assuming 4.5 points improvement in function and 3% improvement in Month 6 survival rate 20 Power gain up to ~8% comparing to N=102 with the potential of saving ~N=27 subjects

Conclusion The proposed historical data borrowing approach ethically reduces patient burden and increases the probability of success The conditional borrowing component is critical for regulatory interactions With conditional borrowing, under the null hypothesis, as the common distribution (TA and CC) in the current trial shift away from the observed distribution of HCP, the Type I error inflation is bounded Further research and simulation are ongoing to refine the proposed historical data borrowing approach 21

References 22