Improving the Quality of Research Reporting: Headache Steps Up to the Plate

Authors

  • Elizabeth W. Loder MD, MPH,

    1. From Harvard Medical School and the Division of Headache and Pain, Department of Neurology, Brigham and Women's/Faulkner Hospitals, Boston, MA USA (E.W. Loder); Department of Psychiatry and Human Behavior, and Director, Head Pain Center, University of Mississippi Medical Center, Jackson, MS USA (D.B. Penzian).
    Search for more papers by this author
  • Donald B. Penzien PhD

    1. From Harvard Medical School and the Division of Headache and Pain, Department of Neurology, Brigham and Women's/Faulkner Hospitals, Boston, MA USA (E.W. Loder); Department of Psychiatry and Human Behavior, and Director, Head Pain Center, University of Mississippi Medical Center, Jackson, MS USA (D.B. Penzian).
    Search for more papers by this author

  • Conflict of Interest: Dr Loder receives salary support from the British Medical Journal in exchange for services as a clinical editor. She is an associate editor of Headache and reports no other conflicts of interest. Dr Penzien reports no conflicts of interest.

Elizabeth W. Loder, MD, MPH Chief of the Division of Headache and Pain, John R. Graham Headache Center, 1153 Centre Street, Suite 4970, Boston, MA 02130, USA.

Good reporting is not an optional extra: it is an essential component of good research . . . we all share this obligation and responsibility.

Professor Douglas Altman, Centre for Statistics in Medicine, University of Oxford, United Kingdom.1

Have you ever read a research article and searched in vain for important details about how the work was conducted? Perhaps you wanted to know how the authors decided on the sample size, whether the subjects in the study were similar to patients you treat, or how many participants dropped out of the study over time. Have you ever searched Medline for the answer to a clinical question and been unable to glean even basic information from the abstracts returned by your search?

Research reports and abstracts should contain sufficient information to meet the needs of their many end users, but often they do not.2 This has bad consequences for doctors, patients, and policymakers who rely on them to make treatment and funding decisions. Other researchers, too, rely on research reports to plan their own projects. And increasingly, previously published research evidence is summarized in systematic reviews and meta-analyses. All of these activities are dependent upon the quality of information in the original research report.

If you have submitted an article to Headache since the beginning of this year, you probably noticed that you were asked to upload a reporting checklist along with your work. In an attempt to improve the quality of research reports in the journal, Headache now requires a completed reporting checklist as a condition of article submission. The electronic manuscript submission system used by the journal has been updated so that the appropriate checklist appears automatically once a prospective author selects a submission category. This change brings our policies in line with those of the leading academic journals.3,4

The published report of a study is the only enduring evidence that the research has been carried out, and of exactly how it has been performed. Good reports should contain a clear explanation of the study methods, describe statistical techniques in enough detail to allow verification of the results from original data, report all results, and interpret and present findings in a balanced and forthright way. If important information is missing from the report, crucial data are lost to posterity. Checklists lessen the chance that something important will be overlooked. They outline the essential features of a good research report and require authors to indicate the page number in their article where each item is discussed. In some cases, they require authors to include flow diagrams showing the flow of participants through the study.

The specific items that need to be included in a research report vary depending upon the design of the study, so there are different checklists for different types of research. A template of the flow diagram required by the CONSORT checklist for randomized controlled trials is reproduced in the Figure. The checklists that will be used by Headache, and the corresponding study types, are listed in the Table. Not every item in a checklist will necessarily apply to every study of that type, so authors are encouraged to employ the checklists as guides to good reporting rather than as inflexible requirements. Authors are free to explain why a particular requirement should not apply and editors may consider their arguments.

Figure Figure.—.

A template of the flow diagram required by the CONSORT checklist.

Table Table.—. Study Types and Corresponding Reporting Checklists
Study typeChecklist of reporting standards
  1. The checklists are available on the Headache website (http://www.headachejournal.org/) and the Equator Network website (http://www.equator-network.org/).

Randomized controlled pharmacotherapy trialsCONSORT – Consolidated Standards of Reporting Trials
Other pharmacotherapy and herbal medicinal trials (noninferiority trials, pragmatic trials, cluster trials, reporting of harms)CONSORT extensions (tailored versions of the main CONSORT Statement produced by the CONSORT Group)
Trials examining behavioral and nonpharmacological interventionsBehavioral/Nonpharmacological Clinical Trials Checklist for Headache (an unofficial extension of the CONSORT Statement and extension adapted from Guidelines for Trials of Behavioral Treatments for Recurrent Headache)6
Observational epidemiology studies (cross-sectional, case–control, cohort)STROBE – strengthening the reporting of observational studies in epidemiology
Diagnostic Accuracy StudiesSTARD – standards for reporting diagnostic accuracy
Systematic reviews and meta-analyses of controlled trialsPRISMA (formerly known as QUOROM) – improving the quality of reports of meta-analyses of randomized controlled trials
Meta-analyses of observational studiesMOOSE – meta-analysis of observational studies in epidemiology
Quality improvement reportsSQUIRE – standards for quality improvement reporting excellence
Qualitative researchCOREQ – consolidated criteria for reporting qualitative research

The items included in most checklists reflect reporting guidelines that have been developed carefully by groups of experts, then field-tested and refined. The first iteration of the CONSORT guidelines and checklist for randomized trials, for example, was developed in the mid-1990s and has undergone a number of subsequent revisions and extensions.5

Headache's editorial staff recently produced a checklist for manuscripts reporting trials of behavioral and nonpharmacologic therapies (Appendix I). The new checklist is based upon the Guidelines for Trials of Behavioral Treatments for Recurrent Headache published under the auspices of the American Headache Society,6 and it incorporates the CONSORT Statement as well as the CONSORT Group's recently published extension to randomized trials of nonpharmacologic treatments. As there was no involvement of the CONSORT group in its development, this checklist is released as the latest addition to the list of unofficial extensions of the CONSORT Statement. This checklist is suited for reporting trials of any design (not only randomized trials) and incorporates many of the research recommendations specific to the headache field, as specified in the behavioral trials guideline.

The benefits of reporting requirements are not theoretical: good evidence shows that checklist use improves the quality of published medical research.7,8 Like the checklists used by pilots, anesthesiologists, and others engaged in high-stakes activities, the routine use of research reporting checklists by scientific and medical journals has proved to be an important component of quality control. Headache is proud to be the first specialty headache journal to step up to the plate and adopt these rigorous standards.

Elizabeth W. Loder, MD, MPH; Donald B. Penzien, PhD

Appendix

APPENDIX I

Behavioral/Nonpharmacological Clinical Trials Checklist for Headache

Checklist of items to include in reports of clinical trials evaluating behavioral/nonpharmacological therapies. An unofficial extension of the CONSORT Statement (and extension to randomized trials of nonpharmacologic treatments) adapted from Guidelines for Trials of Behavioral Treatments for Recurrent Headache

Section and topicItem no.DescriptorPage no.
  • Version 1.1, January, 2008.

  • Penzien DB, Andrasik F, Freidenberg BM, Houle TT, Lake AE, Lipchik GL, Holroyd KA, Lipton RB, McCrory DC, Nash JM, Nicholson RA, Powers SW, Rains JC, Wittrock DA. Guidelines for trials of behavioral treatments for recurrent headache, first edition: American Headache Society Behavioral Clinical Trials Workgroup. Headache. 2005;45:S109-131. Checklist developed by Donald B. Penzien, PhD, Timothy T. Houle, PhD, Jeanetta C. Rains, PhD, and Jason L. Roberts, PhD.

  • Number in brackets represents a parallel item in the Consort Statement 2001 Checklist and extension to nonpharmacologic treatment (Boutron I, Moher D, Altman DG, Schulz KF, Ravaud P; CONSORT Group. Extending the CONSORT Statement to randomized trials of nonpharmacologic treatment: explanation and elaboration. Ann Intern Med. 2008;148(4):295-309; Boutron I, Moher D, Altman DG, Schulz KF, Ravaud P; CONSORT Group. Methods and processes of the CONSORT Group: Example of an extension for trials assessing nonpharmacologic treatments. Ann Intern Med. 2008;148(4):W60-66.

  • §

    Headache Classification Subcommittee of the International Headache Society. The international classification of headache disorders (2nd Ed.). Cephalalgia. 2004;24(Suppl. 1):1-160.

  • A strategy for analysis of data in which all participants are included in the group to which they were assigned whether or not they completed the intervention given to the group.

  • ††

    A strategy for analysis of data in which only participants who completed a particular phase or the entire intervention are included in the analysis of group data.

  • ‡‡

    See statistical reporting checklists in Houle TT; Penzien DB. Statistical reviewing for Headache. Headache. 2009;49:159–161.

Abstract1(a) principal research question(s) and design [C1] 
(b) balanced summary of methods and principal findings 
Introduction2(a) scientific background and rationale for study [C2] 
(b) research objectives and hypotheses [C5] 
Methods selection of patients3(a) patient eligibility criteria and settings and locations where data were collected [C3] 
(b) sources of patients 
(c) recruitment methods 
(d) inducements for participation 
(e) whether or not patients sought treatment 
(f) diagnosis using ICHD-II diagnostic criteria§ and chronicity 
(g) baseline symptom frequency and severity criteria 
(h) criteria for pertinent medical and psychiatric comorbidities 
Trial design and execution4(a) research design (eg, case report, single case experimental design, single-group outcome study, parallel group outcome study) and control conditions (if implemented) 
(b) method for sample size determination, and when applicable, (a) explanation of stopping rules of interim analyses, and (b) details of any clustering by centers or care providers [C7] 
(c) method for allocating patients to conditions (eg, randomization, stratification) [C8, C9, C10, C11] 
(d) whether patient, therapist, and outcome assessor were blind to intervention condition (note: double-blinding behavioral and many nonpharmacological treatment and control conditions is rarely, if ever, practical or possible) [C9, C11a, C11b] 
(e) duration of baseline and outcome assessment periods, treatment periods, and schedule of follow-up assessments 
(f) eligibility criteria for intervention setting as well as qualifications, training, and experience of research staff and practitioners [C3] 
(g) details of therapeutic interventions sufficient to allow replication (eg, standardization of interventions, tailoring interventions to individuals, when interventions were administered) [C4, C4a, C4b] 
(h) how treatment integrity was assessed or enhanced [C4c] 
(i) details of any concomitant medication and dietary supplement use
(j) clearly defined primary and secondary outcome measures (including details of reliability and validity) [C6] 
(k) daily headache diary as principal measure for assessing treatment outcome; report of multiple headache variables (eg, intensity, duration) that includes a measure of headache frequency 
(l) standardized measures of disability, functional status, and/or “quality of life” to include at least one headache-specific measure 
(m) statistical methods presented in sufficient detail to allow replication [C12] 
(n) oversight by institutional review board or equivalent ethics committee 
Results5(a) flow of patients through each stage of study (flow diagram recommended) including number of patients assessed, enrolled, allocated to each condition, treated by each center or care provider, lost to follow-up, and included in each analysis [C13, C16] 
(b) dates defining periods of recruitment and follow-up [C14] 
(c) baseline clinical characteristics and demographics by condition, center, and care provider [C15] 
(d) findings for primary and secondary outcome measures by condition [C17] 
(e) “intention-to-treat” as well as “completer”†† analyses for principal dependent measures (at a minimum) [C16, C18] 
(f) optimal reporting of statistics (eg, choice of test, effect size, n/df, variance, significance level, and value)‡‡ 
(g) treatment responder rate by condition (ie, proportion of patients deemed “clinically improved”) 
(h) adverse events by condition; whether or not an adverse event led to discontinuation of treatment [C19] 
Discussion6(a) key findings with reference to research objectives and hypotheses [C20] 
(b) conservative interpretation of results in the context of extant literature; implications of findings and their generalizability (external validity) [C21, C22] 
(c) study limitations [C20] 
Other7(a) specify sources of financial and other important support for the research 
(b) disclose potential conflicts of interest 

Ancillary