Evaluating the evidence: the methodological and reporting quality of comparative observational studies of surgical interventions in urological publications
Article first published online: 19 NOV 2008
© 2008 THE AUTHORS. JOURNAL COMPILATION © 2008 BJU INTERNATIONAL
Volume 103, Issue 8, pages 1026–1031, April 2009
How to Cite
Tseng, T. Y., Breau, R. H., Fesperman, S. F., Vieweg, J. and Dahm, P. (2009), Evaluating the evidence: the methodological and reporting quality of comparative observational studies of surgical interventions in urological publications. BJU International, 103: 1026–1031. doi: 10.1111/j.1464-410X.2008.08155.x
- Issue published online: 26 MAR 2009
- Article first published online: 19 NOV 2008
- Accepted for publication 12 August 2008
- evidence-based medicine;
- research design standards;
- publishing standards;
- observational study methods
To develop and apply a standardized evaluation form for assessing the methodological and reporting quality of observational studies of surgical interventions in urology.
An evaluation standard was developed using the Consolidated Standards for Reporting Trials statement and previously reported surgical reporting quality instruments. Consensus scoring among three reviewers was developed using two distinct sets of studies. All comparative observational trials involving therapeutic surgical procedures published in four major urological journals in 1995 and 2005 were randomly assigned to each reviewer. Categories of reporting adequacy included background, intervention, statistical analysis, results and discussion.
Twenty-seven articles in 1995 and 62 in 2005 met the inclusion criteria; 90% of studies were retrospective. From 1995 to 2005, the overall reporting quality score increased by 3.9 points (95% confidence interval, CI, 2.7–5.9; P = 0.001), from a mean (sd) of 19.1 (3.9) to 23.0 (4.2) on a scale of 0–42. There were significant improvements in the reporting categories of study background (+0.7 points, 95% CI 0.1–1.3, P = 0.043, 0–8-point scale), intervention (+1.6 points, 0.8–2.3, P = 0.001, 0–9-point scale), and statistical analysis (+0.8 points, 0.2–1.4, P = 0.006, 0–9-point scale). There were smaller and statistically insignificant improvements for results (+0.5 points, −0.3 to 1.2, P = 0.217, 0–10-point scale) and discussion reporting (+0.4 points, −0.1 to 0.8, P = 0.106, 0–6-point scale).
There have been minor improvements in the reporting of observational studies of surgical intervention between 1995 and 2005. However, reporting quality remains suboptimal. Clinical investigators, reviewers and journal editors should continue to strive for transparent reporting of the observational studies representing the bulk of the clinical evidence for urological procedures.