Reporting statistical results


This editorial draws attention to an initiative of The Physiological Society, which has commissioned a series of articles on best practice in statistical reporting in life science journals. The articles are published in the two journals of The Physiological Society, Journal of Physiology and Experimental Physiology. Other journals including the British Journal of Pharmacology and Advances in Physiology Education are joining this project. In an earlier initiative, The American Physiological Society published guidelines for the reporting of statistical results (Curran-Everett & Benos, 2004, 2007). These guidelines have partly influenced the section on reporting statistical results in the latest version of Instructions for Authors for the Journal of Fish Biology. In Curran-Everett & Benos (2004), 10 guidelines are provided and these are listed below with some commentary.

  • 1If in doubt and if possible, consult a statistician at the stage of experimental or survey design. This will ensure that the data collected can be correctly analysed by accepted statistical methods.
  • 2Chose a critical significance level α appropriate to aims of the research. Conventionally, an α < 0·05 is used to define significance level, but there may be circumstances when α < 0·1 or <0·01 are more appropriate.
  • 3Name the statistical methods used, citing the textbooks or review papers that are the sources of the methods. Separately, identify the commercial software used for the analyses and quote the URL.
  • 4Use an accepted method to control for multiple comparisons. If there is only a single dependent variable, the best solution is to use pre-planned orthogonal contrasts. The use of methods of multiple comparisons such as Tukey's HSD test, however, may be acceptable (Quinn & Keough, 2002). If there are several dependent variables measured on the same replicate individuals, the situation is more complicated. A common solution is to apply a Bonferroni correction, but the use of false discovery rate is increasing (Curran-Everett, 2000).
  • 5Report variability in a sample using the standard deviation (s.d.) rather than a standard error (s.e.). The s.d. describes the variability of the observed data, whereas the s.e. describes the theoretical dispersion of sample means about the population mean.
  • 6Use confidence intervals (CIs) to describe the uncertainty about the true value of a population parameter. The 95% CI is typically used, but in some circumstances, a 99% CI may be more appropriate. The CI may be reported in the text, tables or graphics. The 95% CI about a population parameter allows the claim that with 95% confidence, the true population parameter lies in the defined interval. In contrast, even with a relatively high number of replicates, the s.e. of the mean only gives about a 68% CI. Confidence intervals should be provided for sample statistics such as means and regression coefficients.
  • 7Report a precise probability (P) value. Most computer software packages now give precise P-values. Such precise values are useful for researchers carrying out meta-analyses. This guideline, however, is more contentious. The precise P-value reported by a computer package assumes that the data analysed conform precisely to all the assumptions of the statistical method used. This may not be the case with biological data and then it is better to report a P-range, e.g. 0·05 > P > 0·01. The problem can be circumvented by providing the statistical method, the value of the test statistic, degrees of freedom (d.f.) and the P-value range (e.g. ANOVA: Fn1,n2 = x; 0·05 > P > 0·01). If a reader wishes to determine the precise P-value (hence making a strong assumption about the data), the value can be obtained from the test statistic and associated d.f. The information in this form is also valuable for a reviewer, allowing a check that the use of the test statistic is correct.
  • 8The number of digits used when reporting a quantity should reflect the accuracy of the original measurements and be commensurate with scientific relevance of the measurements. Computer packages frequently give an excessive number of digits when presenting results and the number of digits should be adjusted carefully before being included in a report. Note that for the Journal of Fish Biology means and their errors must have the same number of decimal places.
  • 9In the ‘Abstract’ of a manuscript provide confidence intervals and preciseP-values for each result reported. This guideline is not followed by the Journal of Fish Biology.
  • 10Use the numerical bounds of the confidence interval and the P-value to interpret the main results. In general, there needs to be a shift of attention from the level of significance to the magnitude of the effect declared significant (Nakagawa & Cuthill, 2007). For example, pay particular attention to the r2-value in regression analyses. Is the significant regression accounting for a high proportion of the variance in the dependent variable?

In parallel with these guidelines, a series of useful articles on statistical methods have been published by Curran-Everett in Advances in Physiology Education and can be downloaded as PDF files. A discussion of data exploration in the context of statistical analyses is also a valuable resource (Zuur et al., 2010).

R. J. Wootton, Assistant Editor J. F. Craig, Editor-in-Chief