Study Size and Power Calculations in Research
This content discusses the importance of determining the appropriate study size and power calculations in observational comparative effectiveness research. It covers key factors such as hypothesis specification, statistical software, precision expectations, and study feasibility. Understanding these aspects is crucial for ensuring meaningful and reliable research outcomes.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Study Size Planning for Observational Comparative Effectiveness Research Prepared for: Agency for Healthcare Research and Quality (AHRQ) www.ahrq.gov
Outline of Material This presentation will: Describe all relevant assumptions and decisions Specify the type of hypothesis, the clinically important inferiority margin or minimum clinically important excess/difference, and the level for the confidence interval Specify the statistical software and command or the formula to calculate the expected confidence interval Specify the expected precision (or statistical power) for any subgroup analyses Specify the expected precision (or statistical power) as sensitivity analyses in special situations
Introduction Study feasibility relies on whether the projected number of accrued patients is adequate to address the scientific aims of the study. Many journal editorial boards endorse reporting of study size rationale. However, this rationale is often missing from study protocols and proposals. Interpreting study findings in terms of statistical significance in relation to the null hypothesis implies a prespecified hypothesis and adequate statistical power. Without the context of a numeric rationale for the study size, readers may misinterpret the results.
Study Size and Power Calculations in Randomized Controlled Trials (1 of 3) Reporting on study size rationale in the study protocol is often required by institutional review boards before data collection can begin. The rationale for study size depends on calculations of the study size needed to achieve a specified level of statistical power. Statistical power is defined as the probability of rejecting the null hypothesis when an alternative hypothesis is true. Software packages and online tools can assist with these calculations.
Study Size and Power Calculations in Randomized Controlled Trials (2 of 3) Specify the clinically meaningful or minimum detectable difference. Identify the size of the smallest potential treatment effect that would be of clinical relevance. Calculate the study size, assuming the value represents the true treatment effect. Specify a measure of data variability. For continuous outcomes, make assumptions about the standard deviation. For occurrence of event outcomes (e.g., death), estimation of the assumed event rate in the control group is necessary.
Study Size and Power Calculations in Randomized Controlled Trials (3 of 3) Needed study size depends on the chosen type 1 error rate ( ) and required statistical power. Use a conventional statistical significance cutoff of = 0.05 and a standard required power of 80 percent. Consider potential reductions in the number of recruited patients available for analysis. Effect of Interest Therapy 1 Risk Therapy 2 Risk Desired Power Needed Study Size Needed Recruitment Scenario 1 0.75 0.020 0.015 80% 10,795 13,494 2 0.75 0.100 0.075 80% 2,005 2,507 3 0.50 0.100 0.050 80% 435 544 4 0.50 0.100 0.050 90% 592 728 An example of adequately reported consideration of study size under several potential scenarios that vary the baseline risk of the outcome, the minimum clinically relevant treatment effect, and the required power.
Considerations for Observational Comparative Effectiveness Research Study Size Planning Sample size and power calculations in the context of randomized controlled trials are relevant for observational studies, but their application may differ. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guidelines Funding agencies often ask for statistical power calculations, while journal editors ask for confidence intervals.
Considerations That Differ From Nonrandomized Studies Confounding bias, measurement error, and other biases should concern investigators more than the expected precision when they consider the feasibility of an observational comparative effectiveness study. Controlling for confounding can also reduce the precision of estimated effects (often seen in studies with propensity score matching). Retrospective studies often suffer from a higher frequency of missing data, which can limit precision and power.
Conclusions To ensure adequate study size and appropriate interpretation of results, provide a rationale for study size during the planning and reporting stages. All definitions and assumptions should be specified, including primary study outcome, clinically important minimum effect size, variability measure, and type I and type II error rates. Consider loss to followup, reductions due to statistical methods to control for confounding, and missing data to ensure the sample size is adequate to detect clinically meaningful differences.
Summary Checklist (1 of 2) Guidance Key Considerations Describe all relevant assumptions and decisions. Report the primary outcome on which the study size or power estimate is based. Report the clinically important minimum effect size (e.g., hazard ratio 1.20). Report the type I error level. Report the statistical power or type II error level (for study size calculations) or the assumed sample size (for power calculations). Report the details of the sample size formulas and calculations including correction for loss to followup, treatment discontinuation, and other forms of censoring. Report the expected absolute risk or rate for the reference or control cohort, including the expected number of events. Specify the type of hypothesis, the clinically important inferiority margin or minimum clinically important excess/difference, and the level of confidence for the interval (e.g., 95%). Types of hypotheses include equivalence, noninferiority, and inferiority.
Summary Checklist (2 of 2) Guidance Key Considerations Examples include Stata , Confidence Interval Analysis, and Power Analysis and Sample Size (PASS). Specify the statistical software and command or the formula to calculate the expected confidence interval. Specify the expected precision (or statistical power) for any planned subgroup analyses. Specify the expected precision (or statistical power) as sensitivity analyses in special situations. Special situations include: The investigators anticipate strong confounding that will eliminate many patients from the analysis (e.g., when matching or trimming on propensity scores). The investigators anticipate a high frequency of missing data that cannot (or will not) be imputed, which would eliminate many patients from the analysis.
