Search methods for identification of studies
See search methods used in reviews by the Cochrane Collaborative Review Group on HIV Infections and AIDS.
We formulated a comprehensive search strategy to identify all relevant studies regardless of language or publication status (published, unpublished, in press and in progress). Full details of the Cochrane HIV/AIDS Review Group methods are published in the section on Collaborative Review Groups in The Cochrane Library.
We searched the following electronic databases from the period of 1 January 1980 through 23 December 2013:
CENTRAL (Cochrane Central Register of Controlled Trials)
Web of Science
World Health Organization (WHO) Global Health Library (http://www.globalhealthlibrary.net), which includes references from AIM (AFRO), LILACS (AMRO/PAHO), IMEMR (EMRO), IMSEAR (SEARO), and WPRIM (WPRO)
Searching other resources
We searched the Aegis archive of HIV/AIDS conference abstracts, which includes abstracts for the following conferences up to 2008:
Conferences on Retroviruses and Opportunistic Infections (CROI)
International AIDS Society, International AIDS Conferences (IAC)
International AIDS Society, Conferences on HIV Pathogenesis, Treatment and Prevention (IAS)
We searched the conference web sites for abstracts from 2008 through 2013. Additionally, we examined the references of included studies and reviews that we identified.
Data collection and analysis
The methodology for data collection and analysis was based on the guidance of Cochrane Handbook of Systematic Reviews of Interventions (Higgins 2008).
Selection of studies
One author (THH) performed a broad first examination of all downloaded material from the electronic searches to exclude citations that were plainly irrelevant. Two authors (HB, THH) read the titles, abstracts and descriptor terms of the remaining downloaded citations to identify potentially eligible reports. Full text articles were obtained for all citations identified as potentially eligible, and two authors (HB, THH) independently inspected these to establish the relevance of the article according to the pre-specified criteria. If there was uncertainty about the eligibility of the record, the full article was obtained.
Two authors (HB, THH) independently applied the inclusion criteria, and any differences were resolved by discussion. Studies were reviewed for relevance based on study design, types of participants and outcome measures.
Data extraction and management
Two authors (HB, GWR) independently extracted data into a standardised, pre-piloted data extraction form. The following characteristics were extracted from each included study:
Administrative details: trial identification number; author(s); published or unpublished; year of publication; number of studies included in paper; year(s) in which study was conducted; details of other relevant papers cited
Details of the study: study design; type, duration and completeness of follow-up; location/orientation of study (e.g. higher-income vs. low or middle-income country; stage of HIV epidemic)
Details of participants: age range, sex, or sexual orientation if appropriate; clinical characteristic if appropriate, risk for HIV infection, risk for YF
Details of intervention: venue; stage of HIV infection when given YF vaccine
Details of outcomes
Details necessary for risk of bias or methodological quality assessment
Assessment of risk of bias in included studies
Two review authors (HB, GWR) independently assessed risk of bias for each study using the bias assessment tool described in the Cochrane Handbook (Higgins 2008). We resolved any disagreement by discussion or by involving a neutral third party.
The Cochrane approach assesses risk of bias in individual studies across six domains: allocation concealment, blinding, incomplete outcome data, selective reporting, sequence generation and other forms of potential bias.
Allocation concealment (checking for selection bias)
Adequate: participants and the investigators enrolling participants cannot foresee assignment
Inadequate: participants and investigators enrolling participants can foresee upcoming assignment (e.g., an open random allocation schedule, a list of random numbers), or envelopes were unsealed, non-opaque or not sequentially numbered
Unclear: insufficient information to permit judgement of the allocation concealment or the method not described.
Blinding (checking for performance bias and detection bias)
Adequate: blinding of the participants, key study personnel and outcome assessor and unlikely that the blinding could have been broken. Not blinding in the situation where non-blinding is unlikely to introduce bias.
Inadequate: no blinding or incomplete blinding when the outcome is likely to be influenced by lack of blinding.
Unclear: insufficient information to permit judgment of adequacy or otherwise of the blinding.
Incomplete outcome data (checking for possible attrition bias through withdrawals, dropouts, protocol deviations)
Adequate: no missing outcome data, reason for missing outcome data unlikely to be related to true outcome or missing outcome data balanced in number across groups.
Inadequate: reason for missing outcome data likely to be related to true outcome, with either imbalance in number across groups or reasons for missing data.
Unclear: insufficient reporting of attrition or exclusions.
Adequate: a protocol is available which clearly states the primary outcome is the same as in the final trial report.
Inadequate: the primary outcome differs between the protocol and final trial report.
Unclear: no trial protocol is available or there is insufficient reporting to determine if selective reporting is present.
Sequence generation (checking for selection bias)
Adequate: investigators described a random component in the sequence generation process, such as the use of random number table, coin tossing, card or envelope shuffling.
Inadequate: investigators described a non-random component in the sequence generation process, such as the use of odd or even date of birth, algorithm based on the day or date of birth, hospital or clinic record number.
Unclear: insufficient information to permit judgment of the sequence generation process
Other forms of bias
Adequate: there is no evidence of bias from other sources.
Inadequate: there is potential bias present from other sources (e.g., early stopping of trial, fraudulent activity, extreme baseline imbalance or bias related to specific study design).
Unclear: insufficient information to permit judgment of adequacy or otherwise of other forms of bias.
For blinding and incomplete outcome data, multiple entries would have been made if more than one outcome (or time points) had been involved.
We used the Newcastle-Ottawa Quality Assessment Scale for Cohort Studies (Newcastle-Ottawa) to assess methodological quality in the included non-randomised studies. Specifically, the scale uses a star system to judge three general areas: selection of study groups, comparability of groups and ascertainment of outcomes (in the case of cohort studies). This instrument can thus be used in a systematic review to assess the quality of non-randomised studies.
Assessment of Quality of Evidence Across Studies
We assessed the quality of evidence with the GRADE approach (Guyatt 2008), defining evidence quality for each outcome as “the extent to which one can be confident that an estimate of effect or association is close to the quantity of specific interest” (Higgins 2008). When data for the same outcome from two or more studies can be pooled, evidence quality for that outcome is assessed across the studies providing those data. The quality rating has four levels: high, moderate, low and very low. Data from RCTs are initially considered to provide high quality evidence, but this evidence can be downgraded. Data from non-randomised studies are initially considered to provide low quality evidence, but this evidence can be upgraded or further downgraded. Factors that can decrease evidence quality include limitations in design, indirectness of evidence, unexplained heterogeneity or inconsistency of results, imprecision of results typically due to small numbers of events or high probability of publication bias. Factors that can increase evidence quality include a large magnitude of effect, all plausible confounding leading to an underestimation of effect, or a dose-response gradient.
Measures of treatment effect
We used Review Manager 5.1 (RevMan 2011) provided by the Cochrane Collaboration to prepare the review and for statistical analysis. We summarised dichotomous outcomes for effect using risk ratios (RR), with 95% confidence intervals (CI). We calculated summary statistics and present findings in regard to evidence quality in GRADE summary of findings tables, for all outcomes of interest.
Unit of analysis issues
The unit of analysis was the individual participant.
Dealing with missing data
Study authors would have been contacted to obtain missing data if necessary.
Assessment of reporting biases
We minimised the potential for publication bias by using comprehensive search strategies, which include searching scientific literature from a wide range of databases, published or unpublished, written in any language.
Meta-analysis would have been conducted when appropriate. Since meta-analysis was not possible, we conducted a narrative synthesis of studies. Data were also presented using the GRADEpro software (GRADEpro 2011). We generated GRADE evidence profiles and summary of findings tables.
Subgroup analysis and investigation of heterogeneity
If data had allowed, we had planned to perform subgroup analyses. These analyses could have included subgroup analyses based on age, degree of immunosuppression, study region, middle-income vs. low-income country, characteristics of key populations or other factors.
If the data had allowed, we had planned to examine the contributions of individual studies to overall heterogeneity by removing them one at a time.