Critical Thinking A tour through the science of neuroscience NEBM 10032/5 – Sample size and statistical power Gillian Currie Centre for Clinical Brain Sciences, University of Edinburgh slides at @CAMARADES_

CAMARADES: Bringing evidence to translational medicine

Justification of the number of animals used in an experiment • Scientists who intend to use animals in research must justify the number of animals to be used, and committees that review proposals to use animals in research must review justification to ensure the appropriateness of the number of animals to be used. • The number of animals to be used is often estimated from previous experience. • Ideally, a simple sample size calculation should be performed.

CAMARADES: Bringing evidence to translational medicine

Sample size • Researchers should calculate the sample size necessary to detect a statistically significant effect. • An experiment that is too small may miss biologically important effects, whereas an experiment that is too large wastes animals. • Investigators frequently err on the side of using too few animals rather than too many resulting in a study that has too little power to detect a meaningful or biologically significant result.

CAMARADES: Bringing evidence to translational medicine

Why use so few animals? • Animal researchers are encouraged to reduce the number of experimental animals to a minimum. • For example, the need to use the minimum number of animals to obtain valid results is expressed in the Animals (Scientific Procedures) Act 1986

• Time and budget constraints

CAMARADES: Bringing evidence to translational medicine

CAMARADES: Bringing evidence to translational medicine

Good laboratory practice “Sample Size Calculation The manuscript should describe how the size of the experiment was planned. If a sample size calculation was performed this should be reported in detail, including the expected difference between groups, the expected variance, the planned analysis method, the desired statistical power and the sample size thus calculated. For parametric data, variance should be reported as 95% confidence limits or standard deviations rather than as the standard error of the mean.” Macleod et al. Stroke 2009; 40(3);e50-2.

CAMARADES: Bringing evidence to translational medicine

ARRIVE guidelines

CAMARADES: Bringing evidence to translational medicine

Reporting “Sample size calculation Sample size calculations were performed in STATA/IC 10 (StataCorp,College Station, Texas, USA) with the sampsi function. Stratified meta-analysis of hypothermia treatment in SHRs (10) reported a normalized mean effect size of 49% [standard deviation (SD) = 28%]. To reject the null hypothesis that pethidine does not attenuate this effect, we predicted a normalized mean effect size of hypothermia in the presence of pethidine of 29%. To achieve power = 0·8 and alpha = 0·05 to detect this difference would require a total of 60 animals.” (10) van der Worp HB et al. Hypothermia in animal models of acute ischaemic stroke: a systematic review and meta-analysis. Brain 2007; 130:3063–74. Sena et al. Int J Stroke. 2013 Apr;8(3):180-5. CAMARADES: Bringing evidence to translational medicine

Reporting of sample size calculations • Few studies identified in systematic reviews reported using a sample size calculation. Sample Size calculation Stroke

3%

MND