Statistical Analysis of Method Transfer Studies in Biotechnology Products

Design and statistical analysis of method transfer studies for biotechnology products Meiyu Shen, Lixin Xu Center for Drug Evaluation and Research, U.S. Food and Drug Administration This presentation reflects the views of the author and should not be construed to represent FDA s views or policies

Outline Method development and its life cycle management Purpose of analytical method transfer studies What parameters compared in analytical method transfer studies Testing materials Analysis Conclusion 2

New analytical method development Parameters evaluated Specificity Linearity Accuracy Precision Limits of detection (LOD) Limits of Quantitation (LOQ) Range 3

Life cycle management of analytical procedures Including, but not limited to Trend analysis on method performance at regular intervals to optimize the analytical procedure to revalidate all or a part of the analytical procedure Development and validation of a new or alternative analytical method A new impurity Method transferred to a new testing site 4

Analytical method transfer studies The purpose of method transfer studies (internal) To determine if the two laboratories provide comparable results across the range of interest. If so, then to transfer a fully validated analytical method from the originating lab to a new lab (receiving lab) Once transferred, the method is suitable for its intended use and can be used to ensure process consistency and meet product specifications 5

Analytical method transfer studies How to achieve the goal? Obtaining the comparative data from method transfer studies Checking the receiving lab s bias (difference between the true value and the mean of the receiving lab) Determining success of implementation of the fully validated analytical method in the receiving lab 6

Important Factors in Method transfer studies Suppose that the same type of instrument from the same manufacturer, same reagents, same experimental conditions, and same testing procedure, we investigate the following factors: operators days runs Replicates lots 7

Key parameters in method transfer studies Mean shift (often incorrectly cited as accuracy) Comparing means of two labs Precision Comparing the standard deviations of two labs Bias (accuracy) 8

Testing materials Is the reference standard appropriate material from which comparative data is obtained for method transfer studies? No Since the method is used to ensure process consistency and meet product specifications 9

Testing materials Multiple lots of a drug product if the assay is used for drug releasing tests Multiple lots of a drug substance if assay is used for measuring the content in drug substance Forced degradation samples or samples of a drug substance or a drug product containing pertinent product-related impurities if the transferred assay is stability indicating 10

Literature review: statistical analysis Many proposals, just name a few here Significance testing approach Comparing the means of two labs by the p-value of rejecting H0: R= S Comment: Discouraging the sponsors to use a large sample size and to obtain more precise measurement Quality control method Checking individual values against the control limit Comment: Not quantitative criteria for decision 11

Literature review: statistical analysis -expected tolerance approach Calculating the tolerance interval in which a proportion ( ) of the receiving laboratory population is expected to fall within, Compares the above tolerance interval to acceptance limits around the mean estimate of the sending laboratory Comments: challenge to define the acceptance limits. -content tolerance interval approach to assure more than 100P% of the individual difference (or percent difference, d) between individual results obtained in the sending laboratory and the receiving laboratory are within the predefined boundary (L, U) with 100(1 - )% confidence level. Two-sided tolerance interval Two one-sided tolerance interval Comments: challenge to define L, U, and P 12

Our proposal: Equivalence test for comparing means of two laboratories Denote the means of the response variable of interest by Rand S , respectively, for the receiving laboratory and the sending laboratory. : or H 0 R S R S : H . Here is a pre-specified constant, also called an equivalence margin. (Equation 1) a R S 13

Challenge of setting equivalence margin for equivalence approach Fixed margin Based on the experts knowledge Different margin for a different assay 1% , a reasonable margin for HPLC Too stringent for bioassay 2.5%, a reasonable margin for a specific bioassay Too liberal for HPLC Wider than specification 2% for drug substance assay Challenge: It is hard to have a number 14

Challenge of setting equivalence margin for equivalence approach Non-fixed margin: a function of assay variability Unified rule for many assays Based on statistical power for rejecting the null hypothesis in the equivalence hypothesis test with a limit number of observations (not exceeding hundreds) All margins sits well within the assay specification. 15

How to obtain the assay variability Long term quality control data Not appropriate, e.g., If there is a stability trend over the time If there is a drift from assay instruments over the time Only good if there is no other confounding factor except operators, days, and runs Hard to meet this criteria Comparative method transfer studies We may estimate the assay variability from studies 16

Statistical analysis for the mean difference of two labs Hypothesis testing (1): H0: S R -c Sor S R c S Ha: -c S< S R <c S where S and R are the mean responses of the sending lab and receiving lab, respectively, and c > 0 is the constant. Equivalence margin c S Value of c Determined from power function of rejecting the above hypothesis if Sis known 17

Power function for the two one-sided tests procedure Let be the probability of rejecting H0 under Ha in Hypothesis testing (1) when S= R= . The power function is: ( P 1 , ( , ) * 1P ) * 1 2 ( ) 1 1 2 1 + ( ) 2 2 n n 4 t * * 1 x x 1 2 = ( + ( / 2 1 / 2 x 2 1 ) ) t t x e dx ( / 2 2 ) 2 / 1 1 1 1 0 + + n n n n 1 2 1 2 - 1= 2=c S n1: # of obs. in receiving lab n2: # of obs. in sending lab : normal cumulative function ( ) 18

Determination of =CS ( C n P 0 / 1 2 Power function: c ) 1 C x C x 0 = = = ( + ( * / 2 1 / 2 x , 0 , , , ) ) t t x e dx ( / 2 2 ) 2 / 2 / 1 n n n C C=0.85 is reasonably chosen such that we can achieve about Power=0.80 Power=0.85 85% power with a sample size 20 0.94 0.99 in the range 20 to 30 per 22 0.90 0.95 laboratory. 24 0.86 0.90 Margin: =0.85 S 26 0.82 0.87 28 0.79 0.83 0.80 19 30 0.86

An example: internal transfer to a new site All equipment moved to the new site Personnel transferred to the new site At least 2 lots >2 analysts and >2 days Reasonable sample size per lab: ~20-30 Margin: e.g., 0.85 S Power to pass equivalence test is about 85% under no true mean difference 20

Statistical analysis for the mean difference of two labs Option 1:Treating 0.85 as a constant Estimating from the sending lab S S + . 0 + 85 . 0 + 85 X X X X Define and = = R S 2 S R S 2 S T T 1 2 2 2 S S R R n n n n R S R S ( ) ( ) Concluding equivalence criteria is met if where is the 1- quantile of t-distribution with degrees of freedom , is the nominal significance level (e.g., 0.05). Inflate both type 1 and 2 error rates T 1 and , T 2 t t ( ) t 21

Statistical analysis (continued) Option 2: Considering 0.85 as a random variable S + X X C X X C T T Define and n R R = = R S S R S S ( ) 1 ( ) 1 1 2 + + + + 2 2 S 2 2 S 2 R 2 S 2 2 S n C C n n C C 0 0 S R S 1 1 n n n Where = C S S S 0 2 2 2 T 1 Concluding equivalence criteria is met if and , where is the 1- quantile of standard normal distribution, is the nominal significance level (e.g., 0.05). Z Z 2 T Z 22

Head-to-head approach for comparing precisions obtained in two labs Hypothesis H0: R S Hypothesis testing: small powers to reject H0 for small samples. Check the point estimate S R 23

Receiving labs bias verification Important to check bias since the equivalence margin can be large enough such that 90% confidence interval in mean differences falls within the equivalence margin but the receiving lab s mean fails the bias criteria. 24

References 1. 2. 3. Pdfs/c1224.pdf. 4. 5. from the Global Bioanalysis Consortium Harmonization Team. The AAPS Journal 2014. 1143-1148 (2014). 6. Wieling J. Robust, fit-for-purpose method transfer: why we should apply equivalence testing. Bioanalysis. 7(7). 807 814 (2015). 7. Lin Z, Li W, Weng N. Capsule review on bioanalytical method transfer: opportunities and challenges for chromatographic methods. Bioanalysis. 3(1). 57 66 (2011). 8. Chambers D, Kelly G, Limentani G, Lister A, Lung KR, Warner E. Analytical Method Equivalency--An Acceptable Analytical Practice. Pharmaceutical Technology. 64-80 (2005). 9. Dew W, Govaerts B, Boulanger B, Rozet E, Chiap P, Hubert P. Using Total Error as Decision Criterion in Analytical Method Transfer. Chemometrics and Intelligent Laboratory Systems. 85.262 268 (2007). 10. Kaminski L, Schepers U, W tzig H. Analytical method transfer using equivalence tests with reasonable acceptance criteria and appropriate effort: Extension of the ISPE concept. Journal of Pharmaceutical and Biomedical Analysis. 53. 1124 1129 (2010). 11. Fr mke C, Hothorn LA, Sczesny F, Onken J, Schneider M. Analytical method transfer: Improving interpretability with ratio-based statistical approaches. Journal of Pharmaceutical and Biomedical Analysis. 74. 186 193 (2013). 12. Zhong JL, Lee K, Tsong Y. Statistical Assessment of Analytical Method Transfer. Journal of Biopharmaceutical Statistics. 18(5). 1005-1012 (2008). 13. Schwenke JR, O'Connor DK. Design and Analysis of Analytical Method Transfer Studies. Journal of Biopharmaceutical Statistics. 18 (5). 1013-1033 (2008). 14. Altan S, Shoung JM. Block designs in method transfer experiments. Journal of Biopharmaceutical Statistics.18(5). 996-1004 (2008). 15. Agut C, Caron A, Giordano C, Hoffman D, S galini A. Transfer of analytical procedures: A panel of strategies selected for risk management, with emphasis on an integrated equivalence-based comparative testing approach. Journal of Pharmaceutical and Biomedical Analysis. 56. 293 303 (2011). 16. Krause S. Validation of Analytical Methods for Biopharmaceuticals: A Guide to Risk-Based Validation and Implementation Strategies. PDA/DHI, River Grove, IL, ISBN-10: 1933722061 (2007). 17. Satterthwaite FE. An Approximate Distribution of Estimates of Variance Components. Biometrics Bulletin. 2. 110 114 (1946). 18. Chen Y, Weng Y, Dong X, Tsong Y. Wald Tests for Variance-Adjusted Equivalence Assessment with Normal Endpoints. Journal of Biopharmaceutical Statistics (2016). http://www.tandfonline.com/doi/full/10.1080/10543406.2016.1265542. 19. Okamoto M. Assay validation and technology transfer: Problems and solutions. Journal of Pharmaceutical and Biomedical Analysis. 87. 308 312 (2014). U.S. Food and Drug Administration. Draft Guidance on Analytical Procedures and Methods Validation for Drugs and Biologics (2014). International Society for Pharmaceutical Engineering (ISPE). The Good Practice Guide: Technology Transfer. ISBN-13: 978-1931879132 (2003). United States Pharmacopeia. Transfer of Analytical Procedure. 37-National Formulary 32. https://hmc.usp.org/sites/default/files/documents/HMC/GCs- Ermer J, Limberger M, Lis K, W tzig H. The transfer of analytical procedures. Journal of Pharmaceutical and Biomedical Analysis. 85: 262 276(2013). Briggs .J, Nicholson R, Vazvaei F, et al. Method Transfer, Partial Validation, and Cross Validation: Recommendations for Best Practices and Harmonization 25

Acknowledgement Dr. Yi Tsong, CDER/OB Dr. Juhong Liu, CDER/OBP Dr. Chikako Torigoe, CDER/OBP 26