Reviewed and revised 26 August 2015
- It is unethical and a waste of time and resources to embark on a study when there is a high chance of a false negative result (Type II error)
- The commonest cause of this is having a sample size that is too small
- The larger the sample size, the more likely it is that the true effect of the intervention will be demonstrated
‘What is the smallest sample I need to be almost certain of producing the true result?’
Definition of power
- Power is the chance of a study successfully demonstrating the ‘true’ result
- it is the probability of detecting a significant difference if one exists
- Power = 1 – the false negative rate
- Power = 1 – beta error
- it is the ability of avoiding a false negative result
- it is the likelihood of correctly rejecting the null hypothesis when it is false
- normally power is 80%, there is a 80% probability of detecting a difference if one exists and a 20% chance of a false negative result
- helps to determine sample size
- need to have adequate power to make sure patients aren’t subjected to risk without cause (time, economic and ethical reasons)
- 3. ensures we don’t recruit too many patients
- 4. adds to publishability
Factors determining sample size
- alpha value = level of significance (normally 0.05, lower alpha requires larger sample size)
- beta-value = power (normally 0.05-0.2, smaller beta/higher the power then the larger sample size required)
- statistical test used (students T if n < 60, normal distribution if n > 60)
- variance of population (the greater the variance, the larger the sample required)
- effect size (the smaller the effect size sought, the larger the sample size required) — this should be based on previous studies, current clinical practice and clinical relevance
An ongoing issue in critical care research is that sample size calculations tend to under-estimate the sample size required because overall ICU mortality is improving.
- power calculations are often based on previous studies, with higher mortality rates in the past
- to have equivalent power, with lower overall mortalities, a larger sample size is typically necessary
- this bias is termed ‘delta inflation’ and results in false negative’ results (Type II error)
References and Links
- Aberegg SK, Richards DR, O’Brien JM. Delta inflation: a bias in the design of randomized controlled trials in critical care medicine. Critical care (London, England). 14(2):R77. 2010. [pubmed] [free full text]
- Arnold BF, Hogan DR, Colford JM Jr, Hubbard AE. Simulation methods to estimate design power: an overview for applied research. BMC Med Res Methodol. 2011 Jun 20;11:94. doi: 10.1186/1471-2288-11-94. Review. PubMed PMID: 21689447; PubMed Central PMCID: PMC3146952.
- Godwin M. Hypothesis: the research page. Part 3: Power, sample size, and clinical significance. Can Fam Physician. 2001 Jul;47:1441-3, 1450-3. Review. English, French. PubMed PMID: 11494932; PubMed Central PMCID: PMC2018536.
- Jones SR, Carley S, Harrison M. An introduction to power and sample size estimation. Emerg Med J. 2003 Sep;20(5):453-8. Review. Erratum in: Emerg Med J. 2004 Jan;21(1):126. PubMed PMID: 12954688; PubMed Central PMCID: PMC1726174.