Assessing the reporting quality of influenza outbreaks in the community

Background High‐quality reporting of outbreak characteristics is fundamental to understand the behaviour of various strains of influenza virus and the impact of outbreak management strategies. However, few studies have systematically evaluated the quality of outbreak reporting. Objectives To conduct a systematic analysis and assessment for reporting quality of influenza outbreaks based on a modified version of the STROBE statement, and to examine characteristics associated with reporting quality. Methods A literature search was conducted across 3 online databases (PubMed, Web of Science, MEDLINE) for reports of influenza outbreaks (pandemic H1N1, avian, seasonal). The quality of reports meeting our eligibility criteria was assessed using the Modified STROBE criteria and assigned a score of 30. Mean differences (MD) and 95% confidence intervals (CI) were reported for comparisons of study characteristics. Results Sixty‐four outbreak reports were available for analyses. The average Modified STROBE score was 20/30. Peer‐reviewed articles were associated with a better quality of reporting (MD 2.79, 95% CI 0.79‐4.78). Likewise, reports from authors affiliated with public health agencies were associated with better quality than those from academic institutions (MD 1.65, 95% CI−0.27‐3.56). Conclusions The development of explicit reporting guidelines specifically geared towards reporting of outbreak investigations proved to be useful. Providing information on patient characteristics, investigation details in introduction and results, as well as addressing limitations that could have biased the findings, were frequently missing in the published reports.


| INTRODUCTION
Influenza causes outbreaks in a broad range of settings including hospitals, schools, long-term care centres and other confined settings. 1 Such outbreaks are largely reported in descriptive studies such as case reports and case series, surveillance reports and cross-sectional studies, and guidelines how these should be reported exist. 2 Objective documentation of outbreak characteristics (eg, infected individuals, setting, duration of exposures potentially associated with outcomes) serves as the primary basis to understand epidemiological characteristics of various strains of influenza virus, and how outbreaks can potentially be managed. 3 For influenza outbreak data to be most informative, it is important that sufficient details are reported, which may be lacking in many reports. 4,5 In fact, few studies have systematically evaluated the quality of outbreak reporting of any type of pathogen. [6][7][8][9] To this end, we sought to conduct a systematic analysis of the quality of reporting of influenza outbreaks and to examine characteristics associated with the quality of reporting.

| METHODS
All decisions regarding eligibility criteria, search strategy, study selection, data collection, quality assessment and analysis were established a priori.

| Eligibility criteria
We included outbreak studies involving human patients only and where the primary outbreak pathogen was either pandemic H1N1, avian or seasonal influenza. However, due to literature including zoonotic diseases (eg, avian influenza), investigations involving animal sources transmitting influenza virus to humans were eligible. We limited reports to those that at least described one or more of the following: onset of outbreak, clinical manifestations, control measures or specific diagnostic testing. Studies that evaluated surveillance systems or developed transmission models were not eligible for inclusion. We also excluded studies that did not provide a descriptive detailed account of individual outbreaks, such as annual surveillance reports.

| Search strategy and data extraction
We searched PubMed, Web of Science and MEDLINE for reports published from 2000 to October 2015 using a basic combination of keywords and subject headings ( Figure 1 and Table S3). Our goal was not to conduct a systematic review of outbreaks but rather to obtain an unbiased sample of more recent influenza outbreaks as a general assessment of its reporting quality. Only English language papers were included.

| Quality assessment
To capture the key elements of outbreak reporting and enable effective assessment of reporting quality, we made several modifications to the original STROBE (Strengthening the Reporting of Observational Studies in Epidemiology, a 22-item checklist) 10 resulting in a 30-item quality assessment tool (referred to as "Modified STROBE" below; Table 1). These changes were made based on detailed discussion amongst the 3 authors. Two of the authors (DM, ML) have content as well as methodological expertise and experience applying STROBE.
With each individual component of the tool worth 1 point, a quality score ("Modified STROBE score") was computed for each outbreak report with a maximum value of 30. Individual Modified STROBE scores for each report are in Table S1, while Table S2 shows modifications made to the original assessment tool.

| Predictor variables
A potential association with the quality of reporting (ie, Modified STROBE score) was assessed for 7 variables: publication year, continent of outbreak, influenza strain, outbreak size, outbreak settings, author affiliation (academic institution vs non-academic [eg, public health agencies and authorities]) and publication type (peer-reviewed vs epidemiologic report). The predictor variables were selected a priori. We examined publications prior to and after 2009 on the basis of 2009 H1N1 pandemic given the large number of articles that followed the pandemic. Similarly, we sought to see differences by H1N1 vs seasonal or avian influenza. As STROBE (STrengthening the Reporting of OBservational studies in Epidemiology) was developed in North America and Europe, we wanted to assess differences between these and other regions. We included outbreak size to assess differences between small (local teams) vs larger outbreaks that may have national teams involved. We were interested in seeing whether hospital staff fared differently in reporting compared to the community. Along a similar line, we wanted to see whether public health officials reported better/worse compared to academics. We expected better reporting with peer-reviewed publications so aimed to assess this as well.
Covariates with a P-value of <.10 in the univariate analysis were included in the multivariate model. We chose the P < .10 threshold to maintain a balance between screening predictive factors for multivariate analysis but also ensuring adequate exclusion for factors deemed limited correlation with differences in quality scores. All statistical analyses were conducted using SPSS/PASW Version 18 (SPSS Inc., Chicago, IL, USA). In the event that the investigation report involved both academic and public health institutions, the corresponding author of the report was used to determine author affiliation.

| RESULTS
A total of 64 of 174 (37%) studies reviewed in full text met our eligibility criteria and underwent assessment for the quality of reporting. 11-74

| Quality assessment
The mean Modified STROBE score of included studies was 20.0 (standard deviation (SD) ± 3.6) of 30. All studies provided scientific background and context to the reported outbreak (Item 2A) and quantitative data on affected patients/reported outbreaks (2C) (Table 1).
Similarly, more than 90% of the reports included an informative summary (1C), elements of study design (3A), motivations behind reporting (3B), breakdown of study size (3M), clinical significance (5A) and external validity of results (5C). In terms of poorly reported elements, the 3 items least frequently reported were 3N (addressing missing data), 3O (sensitivity analysis) and 4E (provision of risk estimates, odds ratio) at 5%, 0% and 25% of studies, respectively (Table 1). Sensitivity analyses and reporting of risk estimates were not necessarily appropriate for all studies, which explains to a large extent as to why these criteria were only met in a small minority of reports.

| Factors associated with better reporting
Of the 64 available reports, 51 (80%) were published from 2009 and after. These reports had significantly higher Modified STROBE scores than those published prior to 2009 (MD 3.43, 95% CI 0.86 to 6.00, P = .010). Forty-nine reports (77%) were peer-reviewed publications.
We found significantly higher scores with reports published in peerreviewed journals as opposed to public health epidemiologic reports (MD 2.79, 95% CI 0.79-4.78, P = .007). The remaining 5 predictors were not found to be significantly predictive for higher Modified STROBE scores (Table 2).
In the multivariate model, only 4 covariates were retained as per our analysis plan: publication type, author affiliation, publication year and outbreak size. Peer-reviewed journals (P = .034) remained a significant predictor for higher Modified STROBE scores. While not associated with reporting quality in univariate analysis, affiliation with public health institutions (P = .035) was associated with significantly higher scores in the multivariate analysis. Meanwhile, the quality of reports published after 2009 was no longer significantly better than the quality of older reports (P = .076).

| DISCUSSION
On average, the 64 influenza outbreak reports assessed in this study met two-thirds of the quality criteria in our Modified STROBE assessment tool. In multivariate analysis, significantly higher Modified STROBE scores were noted for peer-reviewed articles compared to epidemiologic reports and for those written by public health-affiliated authors compared to academic institutions.  in our "academic institution" group. [14][15][16]30,41,50,51,54 The modification to the STROBE instrument was performed internally, and there was no involvement of representatives of public health agencies or any other stakeholders. This will be considered as a next step to further strengthen the instrument for future use. We also acknowledge that our modification to STROBE was based on face validity, and we did not conduct inter-rater reliability or formal validation studies.
In conclusion, development of explicit reporting guidelines specifically geared towards publication of outbreak investigations might be useful, given there were low to moderate (<70%) rates of reports providing information on patient characteristics, investigation