Search methods for identification of studies
See search methods used in reviews by the Cochrane Collaborative Review Group on HIV Infection and AIDS.
We will formulate a comprehensive and exhaustive search strategy in an attempt to identify all relevant studies regardless of language or publication status (published, unpublished, in press and in progress). Full details of the Cochrane HIV/AIDS Review Group methods and the journals hand-searched are published in the section on Collaborative Review Groups in The Cochrane Library.
Journal and trial databases
We will search the following electronic databases, in the period from 01 January 1991, the year the first HPV L1-VLP-based prophylactic vaccine patent application was filed, to the search date:
CENTRAL (Cochrane Central Register of Controlled Trials)
Web of Science / Web of Social Science
World Health Organization (WHO) Global Health Library, which includes references from AIM (AFRO), LILACS (AMRO/PAHO), IMEMR (EMRO), IMSEAR (SEARO), and WPRIM (WPRO).
Along with appropriate MeSH terms and relevant keywords, we will use the Cochrane Highly Sensitive Search Strategy for identifying reports of randomised controlled trials in MEDLINE, and the Cochrane HIV/AIDS Group's validated strategies for identifying references relevant to HIV infection and AIDS. The search strategy will be iterative, in that references of included studies will be searched for additional references. All languages will be included.
See Appendix 1 for our PubMed search strategy, which will be modified and adapted as needed for use in the other databases.
We will search conference abstract archives on the web sites of the Conference on Retroviruses and Opportunistic Infections (CROI), the International AIDS Conference (IAC), and the International AIDS Society Conference on HIV Pathogenesis, Treatment and Prevention (IAS), for abstracts presented at all conferences through 2012.
Searching other resources
In addition to searching electronic databases, we will contact individual researchers, experts working in the field and authors of major trials to address whether any relevant manuscripts are in preparation or in press. The references of published articles found in the above databases will be searched for additional pertinent materials.
We will search WHO’s International Clinical Trials Registry Platform (ICTRP) to identify ongoing trials.
Data collection and analysis
The methodology for data collection and analysis will be based on the guidance of Cochrane Handbook of Systematic Reviews of Interventions (Higgins 2008). Two authors will independently examine abstracts of all studies identified by electronic or bibliographic scanning. Where necessary, we will obtain the full text to determine the eligibility of studies for inclusion.
Selection of studies
One author will perform a broad first cut of all downloaded material from the electronic searches to exclude citations that are plainly irrelevant. Two authors will read the titles, abstracts and descriptor terms of the remaining downloaded citations to identify potentially eligible reports. We will obtain full text articles for all citations identified as potentially eligible, and two authors will independently inspect these to establish the relevance of the article according to the pre-specified criteria. Where there is uncertainty as to the eligibility of the record, we will obtain and review the full article.
Two authors will independently apply the inclusion criteria, and any differences arising will be resolved by discussion with a neutral arbiter. We will review studies for relevance based on design, types of participants and outcome measures.
Data extraction and management
Two authors will independently extract data into a standardised, pre-piloted data extraction form. The following characteristics will be extracted from each included study:
Administrative details: trial identification number; author(s); published or unpublished; year of publication; number of studies included in paper; year(s) in which study was conducted; details of other relevant papers cited;
Details of the study: study design; type, duration and completeness of follow up; location/orientation of study (e.g. higher-income vs. low or middle-income country; stage of HIV epidemic)
Details of participants: HIV-infected persons, including age, gender, ethnicity, baseline immune status (e.g. CD4 count, HIV viral load), antiretroviral therapy, reported sexual debut status
Details of intervention: HPV vaccine type (bivalent versus quadrivalent), vaccine dosage, comparator of no vaccination or placebo
Details of outcomes: Serum antibodies against HPV 6, 11, 16, or 18; cervical, vulvar, vaginal, penile, or anal infection with HPV type 6, 11, 16, or 18; CIN 2-3, VIN 2-3, VAIN 2-3, or AIN 2-3; number, incidence, frequency, seriousness of adverse events
Details necessary for risk of bias assessment
Assessment of risk of bias in included studies
Two review authors will independently assess risk of bias for each study using the bias assessment tool described in the Cochrane Handbook (Higgins 2008). We will resolve any disagreement by discussion or by involving a neutral third party to adjudicate.
The Cochrane approach assesses risk of bias in individual studies across six domains: sequence generation, allocation concealment, blinding, incomplete outcome data, selective outcome reporting and other potential biases.
Sequence generation (checking for selection bias)
Adequate: investigators described a random component in the sequence generation process, such as the use of random number table, coin tossing, card or envelope shuffling
Inadequate: investigators described a non-random component in the sequence generation process, such as the use of odd or even date of birth, algorithm based on the day or date of birth, hospital or clinic record number.
Unclear: insufficient information to permit judgment of the sequence generation process.
Allocation concealment (checking for selection bias)
Adequate: participants and the investigators enrolling participants cannot foresee assignment (e.g., central allocation; or sequentially numbered, opaque, sealed envelopes).
Inadequate: participants and investigators enrolling participants can foresee upcoming assignment (e.g., an open random allocation schedule, a list of random numbers), or envelopes were unsealed, non-opaque or not sequentially numbered.
Unclear: insufficient information to permit judgment of the allocation concealment or the method not described.
Blinding (checking for performance bias and detection bias)
Adequate: blinding of the participants, key study personnel and outcome assessor and unlikely that the blinding could have been broken. Not blinding in the situation where non-blinding is unlikely to introduce bias.
Inadequate: no blinding or incomplete blinding when the outcome is likely to be influenced by lack of blinding.
Unclear: insufficient information to permit judgment of adequacy or otherwise of the blinding.
Incomplete outcome data (checking for possible attrition bias through withdrawals, dropouts, protocol deviations)
Adequate: no missing outcome data, reasons for missing outcome data unlikely to be related to true outcome or missing outcome data balanced in number across groups.
Inadequate: reason for missing outcome data likely to be related to true outcome, with either imbalance in number across groups or reasons for missing data.
Unclear: insufficient reporting of attrition or exclusions.
Adequate: a protocol is available which clearly states the primary outcome is the same as in the final trial report.
Inadequate: the primary outcome differs between the protocol and final trial report.
Unclear: no trial protocol is available or there is insufficient reporting to determine if selective reporting is present.
Other forms of bias
Adequate: there is no evidence of bias from other sources.
Inadequate: there is potential bias present from other sources (e.g., early stopping of trial, fraudulent activity, extreme baseline imbalance or bias related to specific study design).
Unclear: insufficient information to permit judgment of adequacy or otherwise of other forms of bias
For blinding and incomplete outcome data, multiple entries can be made if more than one outcome (or time points) is involved. For non-randomised studies, we may add domains to assess bias, such as matching or adjustment for confounding variables.
We will assess the quality of evidence across the body of evidence using the GRADE approach (Guyatt 2008), which defines the quality of evidence for each outcome as “the extent to which one can be confident that an estimate of effect or association is close to the quantity of specific interest” (Higgins 2008). The quality rating across studies has four levels: high, moderate, low or very low. Randomised trials are considered to be of high quality but can be downgraded for any of five reasons; similarly, observational studies are considered to be of low quality, but can be upgraded for any of three reasons. The five factors that decrease the quality of evidence are 1) limitations in study design; 2) indirectness of evidence; 3) unexplained heterogeneity or inconsistency of results; 4) imprecision of results and 5) high probability of publication bias. The three factors that can increase the quality level of a body of evidence are 1) large magnitude of effect; 2) if all plausible confounding would reduce a demonstrated effect and 3) the presence of a dose-response gradient.
Observational studies will be assessed for risk of bias using the above criteria, as well as other measures including a 9-point rigor scale for assessing the quality of non-randomised studies (Kennedy 2010a, Kennedy 2010b). We will also consult the Cochrane Handbook's chapter on non-randomised studies (Higgins 2008). If necessary, we will contact the Cochrane Bias Methods Group for additional guidance.
Measures of treatment effect
Two authors will independently analyze the data. For randomised controlled trials, we will calculate the relative risk (RR) for dichotomous outcomes and the 95% confidence interval (CI). For continuous data we will calculate a weighted-mean difference.
We will use Review Manager 5 provided by the Cochrane Collaboration for statistical analysis and GRADEpro software (GRADEpro 2008) to produce GRADE Summary of Findings tables and GRADE evidence profiles. We will summarise dichotomous outcomes for effect in terms of risk ratio (RR), rate ratio and number needed to treat (NNT) with their 95% confidence intervals. We will perform tests for interaction to compare estimates within subgroups using methods described by Altman and colleagues (Altman 2003).
We will summarise rate data for individual studies in terms of rate ratios with their 95% confidence intervals. Standard errors for each estimate will be estimated using methods described by Rothman and colleagues (Rothman 1998).
We will calculate summary statistics using meta-analytic methods and present findings in GRADE Summary of Findings tables and GRADE Evidence Profiles for all outcomes of interest.
Unit of analysis issues
The unit of analysis will be the individual study participant.
Dealing with missing data
We will contact study authors if it is necessary to obtain data missing from published reports.
Assessment of heterogeneity
We will use the I2 statistic to measure heterogeneity among the trials in each analysis. If we identify substantial heterogeneity (I2 greater than 50%), we will explore it by pre-specified subgroup analysis.
If heterogeneity persists, we will present results separately and report reasons for the observed heterogeneity.
Assessment of reporting biases
Where we suspect reporting bias we will attempt to contact study authors and ask them to provide missing outcome data. Where this is not possible, and the missing data are thought to introduce serious bias, we will explore the impact of including such studies in the overall assessment of results by a sensitivity analysis.
We will assess the potential for publication bias for the studies using funnel plots. We will attempt to minimise the potential for publication bias by our comprehensive search strategy that includes evaluating published and unpublished literature.
We will conduct meta-analysis, if appropriate, using Cochrane's Review Manager software (RevMan 2011) and present results using the Mantel-Haenzel rate ratio. We will use both fixed and random effects models and conduct sensitivity analysis to explore differences between the two models. If meta-analysis is not possible, a narrative synthesis of studies will be undertaken. Data will also be presented using the GRADEpro software (GRADEpro 2008). GRADE evidence profiles and summary of findings tables will be generated.
When interventions and study populations are sufficiently similar across the different studies, we will pool the data across studies and estimate summary effect sizes using both fixed- and random-effects models. We intend to compare the estimates from fixed- and random-effects models in an attempt to explore the influence of small-study effects on results of a meta-analysis with intra-study heterogeneity. Specifically, we will estimate the log (risk ratio) for each included study and use the inverse variance method to calculate study weights. The inverse variance method assumes that the variance for each study is inversely proportional to its importance, therefore more weight is given to studies with less variance than studies with greater variance. If the estimates between the two modeling approaches are similar, then we can assume effects from small-studies only slightly affect the intervention's summary estimate. If estimates from random-effects are qualitatively substantially more beneficial than fixed-effects estimates, we will investigate whether the interventions were more effective in smaller studies than in larger studies. If upon reviewing the methodologies of the included studies we conclude that the larger studies were more rigorous, we may consider presenting only results from larger studies in a meta-analysis. As such, we intend to explore potential methodologic reasons for those differences in fixed- or random-effects estimates.
We will summarise the quality of evidence for the studies separately for each outcome for which data are available in GRADE Summary of Findings tables and GRADE evidence profiles (Guyatt 2008).
Subgroup analysis and investigation of heterogeneity
Heterogeneity will be explored by analyses per subgroup, which will be performed for populations or types of interventions that are dissimilar in a meaningful way. These analyses could include subgroup analyses based on the HIV risk group, perinatally-acquired vs sexually acquired, the study region, higher-income vs. low or middle-income country, characteristics of key populations, or other factors.
If possible, we will perform sub-group analysis by baseline immune status (e.g. CD4 count) of study participants and by gender. Heterogeneity will also be explored using further sub-group analyses by setting (middle- or low- versus high-income country) and by HPV vaccine type (quadrivalent versus bivalent). A test for interaction will be performed for each subgroup comparison.
If pooled results are heterogeneous for the selected studies, we will conduct sensitivity analyses to identify studies with outlying results for further examination.