Statistical Issues in Clinical Trials: Key Considerations and Examples

13thAnnual Conference on Statistical Issues in Clinical Trials Cluster Randomized Clinical Trials: Challenges and Opportunities Remarks Alisa J. Stephens-Shields April 12, 2021

Key Considerations Impactful Design Features of Group-Randomized Trials Duration of follow up Design at Individual Level Longitudinal Repeated cross-sectional Level of Intervention At the level of the group At the level of the individual Analytic Alternatives for Efficiency and Robustness Semiparametric methods 2 2

Susceptibility of Effect Estimates to Secular Trends Murray: Effect of time on estimation in SW-GRTs Intervention effect confounded with time Patterns of correlation may vary over time In parallel GRTs with repeated cross-sectional design at individual level Distribution of baseline factors can change over time within clusters and intervention arms Can create imbalance over time that affect intervention effect estimate 3 3

Example: STOP- HPV Trial (Szilagyi & Fiks, 2021) Unit of Randomization: Pediatric practice Level of Intervention: Provider (within practice) Outcome: Missed HPV vaccination opportunity at a visit Performance Feedback Prompts Baseline Arm 1 Communication R Baseline Arm 2 Standard of Care Standard of Care Standard of Care Providers: Participating ( 50% req.) or non- participating 6m 6m 6m 4 4

Example: STOP - HPV Visits classified by type and dose Type: Well or sick Dose: Initial or subsequent Difference in Visit Mix by Arm: Intervention Period - Baseline Arm Intervention - 4.4% + 0.5% Control -3.0% +1.5% Increase in proportion of sick visits in intervention arm attenuated estimated effect Unadj: -1.6% (-4.5%, 1.3%) Adj by standardization: -2.4% (-3.5%, -1.25%) Well, init Well, sub -3.9% -1.5% + 0.8% + 3.2% -0.3% +1.9% Sick, init Sick sub +4.0% +1.6% * Relatively more sick visits in intervention practices over time 5 5

Changes in Cluster or Individual Characteristics Over Time Are changes impacted by the intervention? Unlikely in STOP-HPV Possible in other GRTs To control or not control? What is estimand? 6 6

Level of Intervention and Interference Keele: Complier Average Causal Effect (CACE) and direct and spillover effects not identified in GRTs Requires individual-level definition of compliance If intervention at cluster level, consider individual-level measure of uptake Consideration of noncompliance and interference very new CACE definition in presence of interference? Standard definition of CACE does not consider jointly indexed potential outcomes Cluster-level complier definition? Spillover effects may have multiple components Can go through individuals or higher level 7 7

Multiple Mechanisms of Spillover Practice Direct Effect: A Spillover Effects: B, C, D B Providers C B A D Individuals Intervention Effect: Well, Initial visits -6.8% ( -9.7%, -3.9%) -7.3% (-10.6%, -4.0%) Network features (DeGruttola) can provide insight into mechanisms of spillover All Participating (65%) 8 8

Alternative Analytic Approaches Hughes: Cautionary tale of misspecified random effects Possible large type I error inflation with random effect misspecification Advises to use most general model with all 3 random effects (cluster, time, treatment) May not converge Alternative: Semiparametric approaches leveraging baseline covariates Only make assumptions about part of the data Advantages Robust to variance misspecification Robust to misspecification of covariate model (doubly-robust) At least as efficient as unadjusted approaches asymptotically Disadvantages - variance underestimation in small samples Small sample corrections to sandwich variance Permutation inference 9 9

Acknowledgements STOP HPV Collaborators: Peter Szilagyi Alexander Fiks Russell Localio Abigail Breck Mary Kate Kelly Margaret Wright Robert Grundmeier Christina Albertin Laura Shone Jennifer Steffes Cynthia Rand Chloe Hannon Dianna Abney Greta McFarland Brayan Seixas Gerald Kominiski 10 10

References STOP- HPV Szilagyi, P., Humiston, S.G., Stephens-Shields, A.J., , & Fiks, A.G. Impact of a Communication Intervention for Clinicians on Missed Opportunities for HPV Vaccination. JAMA Pediatrics, in press. Causal Effects with Interference and/or Non-compliance Imai, K., Jiang, Z., & Malani, A. (2020). Causal inference with interference and noncompliance in two-stage randomized experiments. Journal of the American Statistical Association, 1-13. Miles, C. H., Petersen, M., & van der Laan, M. J. (2019). Causal inference when counterfactuals depend on the proportion of all subjects exposed. Biometrics, 75(3), 768-777. Hudgens, M. G., & Halloran, M. E. (2008). Toward causal inference with interference. Journal of the American Statistical Association, 103(482), 832-842. Small, D. S., Ten Have, T. R., & Rosenbaum, P. R. (2008). Randomization inference in a group randomized trial of treatments for depression: covariate adjustment, noncompliance, and quantile effects. Journal of the American Statistical Association, 103(481), 271- 279. Semiparametric Methods for Efficiency Gain in Clinical Trials Using Baseline Covariates Colantuoni, E., & Rosenblum, M. (2015). Leveraging prognostic baseline variables to gain precision in randomized trials. Statistics in medicine, 34(18), 2602-2617. Stephens, A. J., Tchetgen Tchetgen, E. J., & Gruttola, V. D. (2012). Augmented generalized estimating equations for improving efficiency and validity of estimation in cluster randomized trials by leveraging cluster level and individual level covariates. Statistics in medicine, 31(10), 915-930. Moore, K. L., Neugebauer, R., Valappil, T., & van der Laan, M. J. (2011). Robust extraction of covariate information to improve estimation efficiency in randomized trials. Statistics in medicine, 30(19), 2389-2408. Zhang, M., Tsiatis, A. A., & Davidian, M. (2008). Improving efficiency of inferences in randomized clinical trials using auxiliary covariates. Biometrics, 64(3), 707-715. 11 11

Changes in Cluster or Individual Characteristics Over Time Are changes impacted by the intervention? Unlikely in STOP-HPV Possible in other GRTs: Vending Label Sales Study, GRT of impact of signage on purchasing (Roberto) Cluster: Machine Individual: Transaction Outcome: Calories and nutrition of item purchased Possible varying characteristic: machine type/offerings Machine type/offerings associated with nutrition of purchased item May change in response to sales To control or not control? What is estimand? 12 12