Conflict of Interest: Dr Loder receives salary support from the British Medical Journal in exchange for services as a clinical editor. She is an associate editor of Headache and reports no other conflicts of interest. Dr Penzien reports no conflicts of interest.
Improving the Quality of Research Reporting: Headache Steps Up to the Plate
Article first published online: 3 FEB 2009
© 2009 the Authors. Journal compilation © 2009 American Headache Society
Headache: The Journal of Head and Face Pain
Volume 49, Issue 3, pages 335–340, March 2009
How to Cite
Loder, E. W. and Penzien, D. B. (2009), Improving the Quality of Research Reporting: Headache Steps Up to the Plate. Headache: The Journal of Head and Face Pain, 49: 335–340. doi: 10.1111/j.1526-4610.2009.01356.x
- Issue published online: 25 FEB 2009
- Article first published online: 3 FEB 2009
Good reporting is not an optional extra: it is an essential component of good research . . . we all share this obligation and responsibility.
Professor Douglas Altman, Centre for Statistics in Medicine, University of Oxford, United Kingdom.1
Have you ever read a research article and searched in vain for important details about how the work was conducted? Perhaps you wanted to know how the authors decided on the sample size, whether the subjects in the study were similar to patients you treat, or how many participants dropped out of the study over time. Have you ever searched Medline for the answer to a clinical question and been unable to glean even basic information from the abstracts returned by your search?
Research reports and abstracts should contain sufficient information to meet the needs of their many end users, but often they do not.2 This has bad consequences for doctors, patients, and policymakers who rely on them to make treatment and funding decisions. Other researchers, too, rely on research reports to plan their own projects. And increasingly, previously published research evidence is summarized in systematic reviews and meta-analyses. All of these activities are dependent upon the quality of information in the original research report.
If you have submitted an article to Headache since the beginning of this year, you probably noticed that you were asked to upload a reporting checklist along with your work. In an attempt to improve the quality of research reports in the journal, Headache now requires a completed reporting checklist as a condition of article submission. The electronic manuscript submission system used by the journal has been updated so that the appropriate checklist appears automatically once a prospective author selects a submission category. This change brings our policies in line with those of the leading academic journals.3,4
The published report of a study is the only enduring evidence that the research has been carried out, and of exactly how it has been performed. Good reports should contain a clear explanation of the study methods, describe statistical techniques in enough detail to allow verification of the results from original data, report all results, and interpret and present findings in a balanced and forthright way. If important information is missing from the report, crucial data are lost to posterity. Checklists lessen the chance that something important will be overlooked. They outline the essential features of a good research report and require authors to indicate the page number in their article where each item is discussed. In some cases, they require authors to include flow diagrams showing the flow of participants through the study.
The specific items that need to be included in a research report vary depending upon the design of the study, so there are different checklists for different types of research. A template of the flow diagram required by the CONSORT checklist for randomized controlled trials is reproduced in the Figure. The checklists that will be used by Headache, and the corresponding study types, are listed in the Table. Not every item in a checklist will necessarily apply to every study of that type, so authors are encouraged to employ the checklists as guides to good reporting rather than as inflexible requirements. Authors are free to explain why a particular requirement should not apply and editors may consider their arguments.
|Study type||Checklist of reporting standards|
|Randomized controlled pharmacotherapy trials||CONSORT – Consolidated Standards of Reporting Trials|
|Other pharmacotherapy and herbal medicinal trials (noninferiority trials, pragmatic trials, cluster trials, reporting of harms)||CONSORT extensions (tailored versions of the main CONSORT Statement produced by the CONSORT Group)|
|Trials examining behavioral and nonpharmacological interventions||Behavioral/Nonpharmacological Clinical Trials Checklist for Headache (an unofficial extension of the CONSORT Statement and extension adapted from Guidelines for Trials of Behavioral Treatments for Recurrent Headache)6|
|Observational epidemiology studies (cross-sectional, case–control, cohort)||STROBE – strengthening the reporting of observational studies in epidemiology|
|Diagnostic Accuracy Studies||STARD – standards for reporting diagnostic accuracy|
|Systematic reviews and meta-analyses of controlled trials||PRISMA (formerly known as QUOROM) – improving the quality of reports of meta-analyses of randomized controlled trials|
|Meta-analyses of observational studies||MOOSE – meta-analysis of observational studies in epidemiology|
|Quality improvement reports||SQUIRE – standards for quality improvement reporting excellence|
|Qualitative research||COREQ – consolidated criteria for reporting qualitative research|
The items included in most checklists reflect reporting guidelines that have been developed carefully by groups of experts, then field-tested and refined. The first iteration of the CONSORT guidelines and checklist for randomized trials, for example, was developed in the mid-1990s and has undergone a number of subsequent revisions and extensions.5
Headache's editorial staff recently produced a checklist for manuscripts reporting trials of behavioral and nonpharmacologic therapies (Appendix I). The new checklist is based upon the Guidelines for Trials of Behavioral Treatments for Recurrent Headache published under the auspices of the American Headache Society,6 and it incorporates the CONSORT Statement as well as the CONSORT Group's recently published extension to randomized trials of nonpharmacologic treatments. As there was no involvement of the CONSORT group in its development, this checklist is released as the latest addition to the list of unofficial extensions of the CONSORT Statement. This checklist is suited for reporting trials of any design (not only randomized trials) and incorporates many of the research recommendations specific to the headache field, as specified in the behavioral trials guideline.
The benefits of reporting requirements are not theoretical: good evidence shows that checklist use improves the quality of published medical research.7,8 Like the checklists used by pilots, anesthesiologists, and others engaged in high-stakes activities, the routine use of research reporting checklists by scientific and medical journals has proved to be an important component of quality control. Headache is proud to be the first specialty headache journal to step up to the plate and adopt these rigorous standards.
Elizabeth W. Loder, MD, MPH; Donald B. Penzien, PhD
- 1Why we need transparent reporting of health research. Excerpt from a presentation delivered at the launch of the EQUATOR network in June, 2008. Available at: http://www.equator-network.org/index.aspx?o=1173 (accessed December 5, 2008).
Behavioral/Nonpharmacological Clinical Trials Checklist for Headache
Checklist of items to include in reports of clinical trials evaluating behavioral/nonpharmacological therapies. An unofficial extension of the CONSORT Statement (and extension to randomized trials of nonpharmacologic treatments) adapted from Guidelines for Trials of Behavioral Treatments for Recurrent Headache†
|Section and topic||Item no.||Descriptor||Page no.|
|Abstract||1||(a) principal research question(s) and design [C1]‡|
|(b) balanced summary of methods and principal findings|
|Introduction||2||(a) scientific background and rationale for study [C2]|
|(b) research objectives and hypotheses [C5]|
|Methods selection of patients||3||(a) patient eligibility criteria and settings and locations where data were collected [C3]|
|(b) sources of patients|
|(c) recruitment methods|
|(d) inducements for participation|
|(e) whether or not patients sought treatment|
|(f) diagnosis using ICHD-II diagnostic criteria§ and chronicity|
|(g) baseline symptom frequency and severity criteria|
|(h) criteria for pertinent medical and psychiatric comorbidities|
|Trial design and execution||4||(a) research design (eg, case report, single case experimental design, single-group outcome study, parallel group outcome study) and control conditions (if implemented)|
|(b) method for sample size determination, and when applicable, (a) explanation of stopping rules of interim analyses, and (b) details of any clustering by centers or care providers [C7]|
|(c) method for allocating patients to conditions (eg, randomization, stratification) [C8, C9, C10, C11]|
|(d) whether patient, therapist, and outcome assessor were blind to intervention condition (note: double-blinding behavioral and many nonpharmacological treatment and control conditions is rarely, if ever, practical or possible) [C9, C11a, C11b]|
|(e) duration of baseline and outcome assessment periods, treatment periods, and schedule of follow-up assessments|
|(f) eligibility criteria for intervention setting as well as qualifications, training, and experience of research staff and practitioners [C3]|
|(g) details of therapeutic interventions sufficient to allow replication (eg, standardization of interventions, tailoring interventions to individuals, when interventions were administered) [C4, C4a, C4b]|
|(h) how treatment integrity was assessed or enhanced [C4c]|
|(i) details of any concomitant medication and dietary supplement use|
|(j) clearly defined primary and secondary outcome measures (including details of reliability and validity) [C6]|
|(k) daily headache diary as principal measure for assessing treatment outcome; report of multiple headache variables (eg, intensity, duration) that includes a measure of headache frequency|
|(l) standardized measures of disability, functional status, and/or “quality of life” to include at least one headache-specific measure|
|(m) statistical methods presented in sufficient detail to allow replication [C12]|
|(n) oversight by institutional review board or equivalent ethics committee|
|Results||5||(a) flow of patients through each stage of study (flow diagram recommended‡) including number of patients assessed, enrolled, allocated to each condition, treated by each center or care provider, lost to follow-up, and included in each analysis [C13, C16]|
|(b) dates defining periods of recruitment and follow-up [C14]|
|(c) baseline clinical characteristics and demographics by condition, center, and care provider [C15]|
|(d) findings for primary and secondary outcome measures by condition [C17]|
|(e) “intention-to-treat”¶ as well as “completer”†† analyses for principal dependent measures (at a minimum) [C16, C18]|
|(f) optimal reporting of statistics (eg, choice of test, effect size, n/df, variance, significance level, and value)‡‡|
|(g) treatment responder rate by condition (ie, proportion of patients deemed “clinically improved”)|
|(h) adverse events by condition; whether or not an adverse event led to discontinuation of treatment [C19]|
|Discussion||6||(a) key findings with reference to research objectives and hypotheses [C20]|
|(b) conservative interpretation of results in the context of extant literature; implications of findings and their generalizability (external validity) [C21, C22]|
|(c) study limitations [C20]|
|Other||7||(a) specify sources of financial and other important support for the research|
|(b) disclose potential conflicts of interest|