Journal of Advanced Nursing

Cover image for Vol. 73 Issue 11

Edited By: Editor-in-Chief: Roger Watson; Editors: Robyn Gallagher, Mark Hayter, Jane Noyes, Rita Pickler & Brenda Roe

Impact Factor: 1.917

ISI Journal Citation Reports © Ranking: 2015: 9/114 (Nursing (Social Science)); 11/116 (Nursing (Science))

Online ISSN: 1365-2648

Statistical Guidelines


Last updated: September 2016

Statistical Guidelines

JAN's statistical editor Irene Hueter has written an editorial you may find helpful: 'The real odds: is success predicted by the 5 per cent chance failure rule in statistics?'


Study Objective(s)

  • State the study aim and objectives clearly and concisely.
  • Explain any technical terminology used to describe the study aim.
  • Indicate which study objectives are primary, secondary or other objectives (if applicable).

Study Design

  • State clearly which parts of the study were designed as exploratory or confirmatory.
  • Recognize that hypothesis generating and hypothesis testing analyses are different and be clear on which you are doing:
Exploratory Analysis: The study objectives may not always lead to pre-determined hypotheses and tests. The choice of hypotheses may depend on the data. The data analysis may include data exploration and require a more flexible approach that allows for changes in response to accumulating results.

Confirmatory Analysis: The key hypothesis is pre-defined and its choice is independent of the data, follows from the primary study objective, and is subsequently tested, upon completion of the study. The study is adequately controlled. Firm evidence in support of the claims should suffice to answer the primary objectives. The results should be robust (in contrast to being sensitive to outliers, missing values) and reliable.

In this section:
  • Describe the main features of the study design and specify the outcome variables.
  • If appropriate (e.g. if the study contains both, confirmatory and explanatory analyses), state which outcome variables are primary, secondary, and other variables.
  • Describe the statistical analysis clearly. If the analysis methods were specified prior to the statistical analysis (e.g. in a protocol), this should be stated here. If some of the analysis methods were selected after looking at the data, this should be mentioned as well.
  • Describe the study design accurately and with enough detail that someone could reproduce the study.
  • Use study/sampling/data flow charts to describe complicated sampling/dropout/study designs where useful and possible.
Population:

  • Describe the target population of the study and also the eligible, evaluable, per protocol, and intention to treat populations (if applicable). The subjects in the study should mirror the target population as closely as possible.
  • Present an adequate description on the type of sample, the selection mechanism of the sample (e.g. random sample, convenience sample, any stratification used), the pool from which the sample was drawn, the pre-specified inclusion and exclusion criteria, the assignment mechanism (randomized, partially randomized, non-randomized,…) to different treatments or arms, and any blinding techniques used.
  • If a convenience sample was used or the subjects were not randomly assigned to treatments or arms, justify these choices, explain for which population the sample is representative and to which extent the results are valid, applicable, and generalizable, be cautious with the use of statistical analysis that is based on and requires random samples and/or randomization, and fully acknowledge the limitations.

Sample Size and Power:

Describe the expected sample size, and the outcome variables, distributional assumptions, parameters, the effect size, the choice of significance levels and power (if appropriate), upon which this sample size was based. Studies should be powered on the main objective and its appropriate analysis.  


State the method of sample size calculation clearly and justify the assumptions made.


Data Collection

Describe the method of data collection and its specifics (e.g. number of questions and range of response scores of questionnaires, meaning of scores).  
  • Provide a justification of the selected tool or instrument, state as to whether the tool/ instrument has been shown to be valid and reliable, and list references that document evidence in support of their psychometric properties.
  • If it is a newly developed or "modified" tool, provide reasons and address the validity and reliability issues of the tool.
  • If it is a modified version of a previously developed tool, address the issue of copyright.

Statistical Analysis

Describe the main features of the performed statistical analysis clearly (e.g. confidence interval, including degree of confidence, hypothesis tests, including null and alternative hypotheses, level of significance, particular test and test statistic,…), explain the statistical methodology that was used, and provide suitable references to the statistical literature if the statistical method is not elementary and/or its description in the paper is not self-contained.  
  • Use and reference literature reviews for sophisticated techniques such as factor analysis. (If a paper’s content is 80% statistical and the reference list only contains one elementary statistics reference, this tends to show the blind acceptance of someone’s state of statistical ignorance and a reluctance to fix it.)
  • Make sure that the analysis relates to the study objectives.
  • State the statistical methods and tests used to analyze the data together with the results and next to the results.
  • Describe in detail the procedures that were applied to handle missing values and data, any outliers (spell out the definition used for an outlier), multiplicities in hypothesis testing, for example, adjustments for multiplicity to quantify the type I error rate (e.g. Bonferroni, Holm, Hochberg, etc. adjusted p-values), or any other irregularities to which the statistical analysis could be sensitive. Adjustments for multiple testing are required in a confirmatory analysis.
  • Pay close attention to such issues as multicollinearity in multiple regression and multiple comparisons in ANOVA. Use multivariate techniques if appropriate for answering the research question (for example, instead of reporting many correlations with no attempt to draw out "genuine" relationships).
  • Mention deviations from the analysis methods specified prior to the statistical analysis.
  • Make sure that the decision to use parametric/ non-parametric statistics is appropriate, use data transformations if needed, provide justifications, and describe the data transformations.
  • Describe the model assumptions checks that were performed (e.g. test of normality or other distributional assumptions under the null hypothesis of any hypothesis test carried out, goodness-of-fit tests, tests for homoscedasticity of residuals, graphical plots or representations).
  • Specify the software and the version of software used in the analysis.
  • Standard deviations and standard errors should not be presented along with means using the symbol ±; instead, represent these as 'mean (SD)' and 'mean (SE)', respectively.
  • Non-zero p-values should be stated, e.g. state p 0.001 instead of p=0.000.

Results

Make sure that the style and presentation of the analysis results and tables in particular are of sufficiently high quality, and the overall grammar and quality of English are readable and accessible and not a barrier to the reviewer or reader of the manuscript.  
  • Present the results of the main analysis carefully and clearly and explain how the results address the study objectives.
  • Illustrate the main characteristics of the key variables in suitable tables and/or graphical presentations and make sure that the latter are useful and not just duplications of information given elsewhere.
  • Report summary statistics, result summaries, or the quantities associated with a p-value in tables.
  • When p-values associated with statistical tests are tabulated, indicate the particular test statistic, the degree of freedom, sample size, and the precise p-value (e.g. "p=0.003" or "p 0.001"). Provide these in all cases, even when the test results are not favorable. In exploratory studies, confidence intervals are preferred over hypothesis tests, yet p-values may be calculated and utilized to flag specific differences of interest and highlight differences worth further examination in future studies.
  • For the presentation of tables, figures, graphical plots, diagrams, and displays and representations, see the guidelines for authors. Make sure that the format of the cell entries and data results are consistent throughout a particular table and manuscript.
  • Use appropriate measures of central tendency and spread (e.g. no means of highly skewed data) as summary statistics.
  • Summarize the response rates and the number (percentage) of missing and non-missing values in the appropriate tables or results.
  • Report effects of variables in measures that are clinically relevant, for example, report the effect of age in 10 year increments rather than 1 year increments, effect of weight in 10 pound increments rather than 1 pound.
  • Distinguish between statistical and clinical significance. Do not conclude that a non-significant result proves the null hypothesis.
  • Indicate adjustments of p-values together with the results.
  • Include a brief summary of the model assumption checks used to validate the assumptions of the presented models.
  • Present evidence that a measuring instrument or questionnaire is reliable and valid in the study population.

Discussion

Make sure that results derive from the data and analyses described and conclusions arise from the results.  
  • Provide an interpretation of the analyses results of the study.
  • Highlight new findings and contributions.
  • Give justice to the content of the "What is known" and "What is new" sections.

Study Limitations:

  • Describe possible limitations to the study suitably.
  • Understand the limits of generalizability of the study, and describe how the sample does or does not adequately represent the population that is under discussion.
  • Identify any potential sampling bias (such as comparability to control group, representativeness of sample population), violations of inclusion/exclusion criteria, irregularities, or deviation from the planned study conduct that may have occurred. These may include limitations that impair the statistical analysis and/or affect the interpretation of the results.
  • Indicate the observed values for the variables and/or statistics whose values were estimated prior to the study in order to estimate the sample size and state any discrepancies between these estimated and observed values. Explain how these discrepancies may impact the power of a hypothesis test.
  • State how the power of a test may be affected by multiplicity issues.
  • Address (if appropriate) the sensitivity of the analyses results, including p-values, regarding missing values, missing data imputation used, multiplicity issues, and violations of model assumptions that may have occurred.
  • Discuss appropriately how outliers were treated that were likely to over-influence the analysis.

Final General Comment:

  • Exercise statistical judgment at all times.
  • If necessary, consult a trained and/or experienced statistician during the design of the study and then to review the statistical analysis, presentation of the results, and interpretation of the results, particularly, for studies that employ more advanced or complicated statistical methods.


SEARCH

SEARCH BY CITATION