Reporting of Numerical and Statistical Differences in Abstracts

Improving but Not Optimal

Authors

  • Eric Dryver MD,

    1. Received from the Department of Medicine, University of Toronto (ED, JEH); and the Institute for Clinical Evaluative Sciences and the Clinical Epidemiology and Health Care Research Program, Sunnybrook and Women's College Health Sciences Centre (JEH), Toronto, Ontario, Canada.
    Search for more papers by this author
  • Janet E. Hux MD, SM, FRCPC

    Corresponding author
      Address correspondence and requests for reprints to Dr. Hux: Institute for Clinical Evaluative Sciences, G-106, 2075 Bayview Ave., Toronto, Ontario, Canada, M4N 3M5 (e-mail: jan@ices.on.ca).
    Search for more papers by this author

Address correspondence and requests for reprints to Dr. Hux: Institute for Clinical Evaluative Sciences, G-106, 2075 Bayview Ave., Toronto, Ontario, Canada, M4N 3M5 (e-mail: jan@ices.on.ca).

Abstract

OBJECTIVE: The reporting of relative risk reductions (RRRs) or absolute risk reductions (ARRs) to quantify binary outcomes in trials engenders differing perceptions of therapeutic efficacy, and the merits of P values versus confidence intervals (CIs) are also controversial. We describe the manner in which numerical and statistical difference in treatment outcomes is presented in published abstracts.

DESIGN: A descriptive study of abstracts published in 1986 and 1996 in 8 general medical and specialty journals. Inclusion criteria: controlled, intervention trials with a binary primary or secondary outcome. Seven items were recorded: raw data (outcomes for each treatment arm), measure of relative difference (e.g., RRR), ARR, number needed to treat, P value, CI, and verbal statement of statistical significance. The prevalence of these items was compared between journals and across time.

RESULTS: Of 5,293 abstracts, 300 met the inclusion criteria. In 1986, 60% of abstracts did not provide both the raw data and a corresponding P value or CI, while 28% failed to do so in 1Dr. Hux is a Career Scientist of the Ontario Ministry of Health and receives salary support from the Institute for Clinical Evaluative Sciences in Ontario.996 ( P < .001; RRR of 53%; ARR of 32%; CI for ARR 21% to 43%). The variability between journals was highly significant ( P < .001). In 1986, 100% of abstracts lacked a measure of absolute difference while 88% of 1996 abstracts did so ( P < .001). In 1986, 98% of abstracts lacked a CI while 65% of 1996 abstracts did so ( P < .001).

CONCLUSIONS: The provision of quantitative outcome and statistical quantitative information has significantly increased between 1986 and 1996. However, further progress can be made to make abstracts more informative.

Ancillary