Selection of studies
Two authors independently assessed the abstracts of all references identified from the search strategy for inclusion and resolved any disagreements by discussion. We excluded abstracts referring exclusively to limited disease or treatments other than chemotherapy compared with placebo or BSC.
Data extraction and management
We entered the trials that met the selection criteria in a specific database. Foreign papers were translated when necessary to allow full assessment. Two authors independently extracted the data from the included studies using a previously standardised and tested form. We resolved discrepancies by discussion. Because most of the studies retrieved by the searches included both patients with limited and extensive disease, we sought specific information about patients with extensive disease from the principal investigators.
To assess the quality of each study we entered the following data in to the database.
Randomisation methods and allocation concealment.
Participants: description of patients, age, sex, and performance status.
Interventions: types of drugs and doses, cycles and frequency, other concomitant or sequential treatment, and time of follow-up.
Blinding strategies in the follow-up and assessment of the results.
Losses to follow-up and use or not of an intention-to-treat analysis.
Sample size and how this was determined.
Statistical analysis performed.
Outcomes: survival time (medians or means at different periods and confidence intervals), complete and partial tumour responses, drug toxicity, and quality of life.
Assessment of risk of bias in included studies
We assessed bias in accordance with The Cochrane Collaboration's tool for assessing risk of bias (Higgins 2011). For each study we considered the following.
- Sequence generation: a description for allocating interventions to participants based on a random process.
* Low risk of bias: the authors describe a random component in the sequence generation process
* High risk of bias: the authors describe a non-random component in the sequence generation process
* Unclear: there is insufficient information about the sequence generation process to permit judgement of "Low risk" or "High risk"
- Allocation concealment: a description of any method to secure implementation of the random assignments to prevent that allocations could have been foreseen.
* Low risk of bias: the authors describe a central allocation, or sequentially numbered of identical appearance, or sealed envelopes opaque and numbered
* High risk of bias: there is a description of open random allocation, or unsealed or non-opaque envelopes, or alternation, or date of birth
* Unclear: there is insufficient information about the method of concealment to permit judgement of "Low risk" or "High risk"
-Blinding of participants and personnel: a description of measures used to blind study participants and personnel for knowledge of which intervention a participant received. Description of any evaluation of effective blinding was also considered. Blinding was considered important for the outcomes of tumour response, toxicity and quality of life, and less important for survival.
* Low risk of bias: blinding of participants and personnel and unlikely that could have been broken, or no blinding or incomplete blinding, but the review authors judge that the outcome is not likely to be influence by lack of blinding
*High risk of bias: blinding of participants and personnel and likely that the blinding could have been broken with the outcome likely to be influenced by lack of blinding, or no blinding or incomplete blinding, and the outcome is likely to be influenced by lack of blinding
* Unclear: there is insufficient information to permit judgement of "Low risk" or "High risk", or the study did not address blinding
-Blinding of outcome assessors: a description of measures used to blind the assessment of the outcomes
*Low risk of bias: blinding of the outcome assessment and unlikely that the blinding could have been broken, or no blinding but the review authors considered that the outcome measurement is not likely to be influence by lack of blinding
*High risk of bias: blinding of the outcome assessment and likely that the blinding could have been broken with the outcome measurement likely to be influenced by lack of blinding, or no blinding when the outcome assessment is likely to be influence by lack of blinding
*Unclear: there is insufficient information to judge the existence of "Low risk" or "High risk" or the study did not address this kind of blinding
- Incomplete outcome data: a description of the level of completeness of each outcome, including withdrawals, dropouts, and protocol deviations. Number of randomised patients in each group and number of evaluated patients with reasons for attrition reported.
*Low risk of bias: any one of the following. No missing outcome data; missing outcome data balanced in number across groups, with similar reasons for missing data; the proportion of missing outcomes compared with the observed event risk is not enough to have a clinically relevant impact on the intervention effect estimate
*High risk of bias: there is reason for missing outcome data likely to be related to the true outcome, with either imbalance in numbers or in reasons for missing data across groups; the proportion of missing outcomes compared with the observed event risk is enough to produce clinically relevant bias in the intervention effect estimate; "as-treated" analysis done with substantial departure of the intervention received form that assigned at randomisation.
*Unclear: there is insufficient information of attrition/exclusions to permit judgement of "Low risk" or "High risk"
*Low risk of bias: any one of the following. The study protocol is available and all of the study's prespecified outcomes that are of interest in the review have been reported or the study protocol is not available but it is clear that the published report include all expected outcomes, including those that were prespecified
*High risk of bias: not all of the study's prespecified outcomes have been reported or one or more primary outcomes are reported using measurements, analysis methods or subsets of data that were not prespecified; or one or more reported primary outcomes were not prespecified, unless clear justification for their reporting is provided, such as an unexpected adverse effect, or one or more outcomes of interest in the review are reported incompletely so that they cannot be entered in a meta-analysis; the study report fails to include results for a key outcome that would be expected to have been reported for such a study
*Unclear: there is insufficient information to permit judgement of "Low risk" or "High risk"
For each domain, two authors independently made the judgment of risk of bias in accordance with the following classification: Low risk of bias; High risk of bias; Unclear.
Measures of treatment effect
Treatment effect measures had to be computed by means of relative risks (dichotomous data), hazard ratios, and standardised mean differences (continuous data). A narrative description of results was reported in the first-line chemotherapy studies. In the second-line chemotherapy studies it was possible to obtain an overall effect measure of two studies. The third study was reported in a narrative. The statistical analysis was done in line with the recommendations of The Cochrane Collaboration using RevMan 5.2. We pooled the study specific hazard ratio estimates and their standard errors to obtain an overall hazard ratio and its 95% confident interval. We used as the method of meta-analysis the inverse variance with the random-effects model of Der Simonian and Laird to calculate the effect size, because of the clinical and methodological heterogeneity of the studies. Heterogeneity in the effect estimates was quantified with the I2 statistic, which indicates the proportion of the total variation in the estimates that is caused by variation between studies not by chance. We considered I2 < 25% as representing low heterogeneity and I2 > 75% as representing high heterogeneity.
Missing data were dealt with by analysing only the available data and not if the trial authors did not provide them. The potential impact of missing data on the findings of the review were addressed in the discussion section.
Assessment of heterogeneity
In terms of clinical heterogeneity, we explored any kind of variability among studies. We assessed baseline group imbalance as well as other sources of clinical heterogeneity such as different diagnostic methods among the studies and groups, adverse effects, differences in dose of drug, length of follow-up, and characteristics of participants. We explored heterogeneity in the meta-analysis by comparing studies in terms of methodological factors such as concealment of allocation, the definition and measurement of outcomes, as well as in relation to clinical heterogeneity.
Assessment of reporting biases
We took into account the following:
publication bias, by contacting pharmaceutical companies for non-published studies; duplicate publication; reporting of outcomes; no language restrictions for inclusion of trials;
the influence of external funding and commercial interests;
a search of multiple database sources, and consulting trials registries.
Since very few studies were included, we did not explore the use of funnel plots.
Subgroup analysis and investigation of heterogeneity
Subgroup analyses were to be done for subsets of participants (gender, age) or for subsets of studies (low risk versus high risk of bias) as a means of evaluating heterogeneous results, if there were a substantial number of studies.
As it was not possible to perform any subgroup analysis in this review, we reported either differences in study and patient characteristics, the way in which outcomes were defined or measured, protocol requirements, the length of follow-up, or differential losses to follow-up. That is, clinical or methodology heterogeneity.