SEARCH

SEARCH BY CITATION

Keywords:

  • quality randomized controlled trial;
  • adjunctive treatment;
  • cerebral malaria

Summary

  1. Top of page
  2. Summary
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References

Background  Randomized controlled clinical trials (RCTs) of adjunctive treatment to reduce the high-mortality associated with cerebral malaria (CM) have so far failed to show any benefit. This may be due in part to improperly designed and/or conducted trials. Therefore a systematic review of quality of RCTs for the treatment of CM with mortality as either primary or secondary outcome published between 1980 and 2000, was conducted.

Methods  RCTs from the peer-reviewed literature using electronic searches. Methodological quality was assessed using an individual component approach (adequacy of concealment of allocation schedule, generation of allocation sequence, double blinding and analysis of participant as randomized). Sample sizes were recalculated for the ability of reviewed trials to detect 25% and 50% reductions in mortality.

Results  Nine trials satisfied the inclusion criteria and were reviewed. Only two had sufficient power to detect a 50% reduction in mortality, and none could detect a 25% reduction. All the trials had inadequate methodological quality in one or more of the components, although in two trials these deficiencies were few.

Conclusion  There is a need for researchers and donors to ensure proper planning and implementation of RCTs in developing countries. In CM, demonstration of worthwhile reduction in mortality by a single intervention will require a large number of subjects, which a single centre may not be able to recruit.


Introduction

  1. Top of page
  2. Summary
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References

Cerebral malaria (CM) constitutes less than 10% of malaria hospital admissions but accounts for up to 40% of mortality attributed to severe malaria and case fatalities remain depressingly high (WHO 2000). The finding that the more rapid clearance of parasitaemia associated with intramuscular artemether compared to quinine was not associated with improved survival (Hoffman 1996) suggests that reductions in the intolerably high-mortality associated with CM might be achieved by blocking the ‘downstream’ effects that occur as a result of Plasmodium falciparum infection. Such down-stream effects include high levels of tumour necrosis factor (TNF) or other pro-inflammatory cytokines and prolonged convulsions (Hoffman 1996; Lou et al. 2001). Treatments designed to block these effects have collectively been referred to as ‘adjunctive treatments’, a number of which has been tested in randomized controlled trials (RCTs) although none has so far been found to be effective in reducing mortality (Warrell et al. 1982; Hoffman et al. 1988; White et al. 1988; Gordeuk et al. 1992; Di Perri et al. 1995; Van Hensbroek et al. 1996; Kochar et al. 1997; Thuma et al. 1998; Crawley et al. 2000).

The importance of RCTs in defining clinical practice has lead to internationally agreed criteria that define proper conduct and standards of RCTs embodied in the CONSORT Statement (Moher et al. 1995). However, a number of studies have found that RCTs frequently fail to meet these standards. Common problems are inadequate concealment of allocation schedule, inadequate generation of allocation sequence, and inadequacies in blinding methods and failure to enrol an adequate sample size to detect an effect that would be of public health benefit (Freiman et al. 1978; Moher et al. 1994; Schultz et al. 1995; Junker 1998; Kunz & Oxman 1998). In these cases the considerable time and effort to conduct a RCT may be wasted and the results may make the conduct of a more definitive trial ethically difficult, with consequent long or permanent loss of an effective drug. An example is the use of streptokinase in myocardial infarction whose implementation was delayed for 25 years because the results of the original trials were inconclusive (Mulrow 1994).

Given this evidence and the importance of CM as a cause of mortality, this review of RCTs of adjunctive treatment in the prevention of mortality due to CM was undertaken to determine if the conduct of these trials met CONSORT standards.

Methods

  1. Top of page
  2. Summary
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References

Identification of RCTs

It was aimed to identify all RCTs of adjunctive treatment of CM defined by WHO criteria (WHO 2000) with mortality as either primary endpoint or represented in the secondary analysis that were conducted between 1980 and 2000. Searches were conducted through electronic databases of Medline, Pubmed, Highwire, HINARI, Science Direct, Biomedcentral and the Cochrane Library. The online archives of the Lancet, British Medical Journal, New England Journal of Medicine, Journal of American Medical Association and International Journal of Epidemiology were also searched. Initial search terms were: ‘malaria’, ‘cerebral malaria’, ‘ancillary treatment’ and ‘adjunctive treatment’. When trials had been identified in this initial search, additional terms of ‘pentoxifyllin’, ‘phenobarbital’, ‘phenobarbitone’, ‘sedation’, ‘anticonvulsants’, ‘deferoxamine’, ‘dexamethasone’ were added. The full texts of papers meeting the search criteria were used as the primary source of data for the study.

In order to validate that the search had identified all the studies meeting the study criteria, review articles and personal contacts were used to identify any unidentified trials that met the search criteria.

Assessment of trial quality

A standard form was used to extract the data from the full published article on each of the identified studies. Data from each trial were extracted twice and any discordance was resolved by reviewing the original article. The name of the first author and identity numbers 1–9 have been used to identify the studies.

The quality of RCTs was classified as adequate, inadequate or missing according to their reporting of methodology as described by Schultz et al. (1995) and in CONSORT guidelines (Altman et al. 2001; Moher et al. 2001). Criteria were:

  • Concealment by central randomization, numbered or coded bottles or containers held by the pharmacy; serially numbered opaque, sealed envelope, or other descriptions that contain elements convincing of concealment were judged to be adequate. Allocation by alternation, reference to study identity numbers or to date of birth were considered inadequate.
  • Generation of allocation sequence using random number tables, computer random number generator, coin-tossing or shuffling were considered adequate. A report of an alternative method was judged as inadequate.
  • Inclusion of all randomized subjects was considered to be adequate, failure to do so was inadequate.
  • Double blinding was considered adequate when it was clear that neither assessors nor subjects were able to identify the allocated treatment. If it was apparent from the paper that this was not achieved the method was inadequate.
  • Reporting of sample size estimation was considered to be adequate if it could be repeated.

Sample size calculation

Sample sizes were recalculated for each trial to show what would have been the sample size required (assuming 95% confidence and 80% power) to detect a two-sided minimum difference of 25% and 50% in mortality between the control and intervention arms using Stata-8 (Stata Corp, College Rd, Texas). For trials that reported details of how the sample size had been calculated, the expected case fatality rate (CFR) in controls was used in the recalculation. For studies that did not provide details of sample size calculation, the sample size was calculated with parameters of 80% power, 95% confidence and the observed CFR among controls in the trial.

Results

  1. Top of page
  2. Summary
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References

Search results

Nine studies were found that satisfied the inclusion criteria and were reviewed. The trials were published between 1982 and 2000 and recruited 1817 patients of both sexes between 9 months and 78 years of age, 340 (18.5%) of who died. Three studies (3, 4 and 5) evaluated the effect of phenobarbitone, two studies each evaluated the effect of dexamethasone (1 and 2) and deferoxamine (6 and 7). Anti-TNF (8) and pentoxifyllin (9) were evaluated in single studies. Four trials (1, 2, 7 and 8) had mortality as the primary outcome, while the others presented mortality in their secondary analysis. The observed odds of mortality, using mortality among controls as a reference, varied from 0.3 to 2.4. The details of the reviewed studies and re-calculated sample sizes and are shown in the Table 1.

Table 1.  The target minimum reduction in mortality, actual and target sample sizes, and recalculated sample size to detect a minimum of 25% and 50% reductions in mortality of nine reviewed trials of adjunctive treatments for cerebral malaria 1980–2000
Idn.StudyExpected CFR in controls Target mortality reduction† (%)Target sampleActual sampleDeaths in interventionDeaths in controls OR death (controls = 1)Sample to detect 50% reduction*Sample to detect 25% reduction*
  1. * Based on the expected case fatality rate among controls or, if none was quoted, on the observed case fatality rate among controls.

  2. † Minimum reduction in mortality that the trial was designed to detect.

  3. ‡ Assumed target reduction in mortality based on the number of subjects recruited and the observed case fatality rate in controls NR = not reported.

  4. § Unadjusted values calculated for this review.

  5. ¶ Trials had death as primary outcome.

1Dexamethasone-1 (Warrell et al. 198232%841121008/50 (16%)9/50 (18%)0.9 (0.3–2.5)§3601370
2Dexamethasone-2 (Hoffman et al. 198825%80100 434/19 (21%)4/19 (21%)1.0 (0.2–4.8)§4381900
3Phenobarbitone-1 (White et al. 1988)NR94‡NR 488/24 (33%)5/24 (21%)2.3 (1.8–2.9)§3601370
4Phenobarbitone-2 (Kochar et al. 1997)NR56‡NR18529/102 (28%)33/83 (40%)0.6 (0.3–1.9)§3081314
5Phenobarbitone-3 (Crawley et al. 2000)30%5032034030/170 (17%)14/170 (8%)2.4 (1.3–4.6)3481496
6Deferoxamine-(Gordeuk et al. 1992)NR81‡NR 837/42 (16%)9/41 (22%)0.7 (0.4–2.4)3601370
7Deferoxamine (Thuma et al. 1998NR64‡NR35232/175 (18%)19/177 (11%)1.8 (0.9–3.6)7963510
8Anti-TNF (8) (Van Hensbroek et al. 199627.5%3058061060/302 (20%)64/308 (21%)0.9 (0.7–1.8)3901682
9Pentoxifyllin (Di Perri et al. 1995)20%25100 56  0/26 (0%)  5/30 (17%)05722504

Qualitative analysis of studies

Concealment of allocation schedule.  Seven (78%) studies had adequate concealment of allocation. Four of these (1, 2, 3 and 8) used identically numbered envelopes or boxes and both intervention and placebo were identical in appearance, while three studies (5, 6 and 7) used pharmacy-held codes as their method of concealment. One study (4) was open and one did not report a method of concealment (9).

Generation of allocation sequence.  Eight (89%) studies did not report an adequate method for generation of allocation sequence. Descriptions judged to be inadequate were: ‘sequential randomization’ (1 and 6), ‘pair’ or ‘block’ randomization (3, 7 and 8), ‘blind drawing’ from envelope (2), ‘sequentially numbered register’ (5) and ‘alternate day allocation’ (4) were used to describe generation of sequence. Only trial 9 used a random number table.

Inclusion in the analysis of all randomized participants.  Eight (89%) studies included all randomized participants in the analysis. Trial 2 enrolled 43 patients for logistic reasons but analysed 38 with equal groups of 19 participants. The reason for the exclusion of five participants was not stated.

Double blinding.  Seven (78%) studies described double blinding. Five of these (1, 2, 3, 5 and 8) described in details what they did. Trials 4 and 9 were conducted in a non-blinded manner.

Justification of sample sizes.  Five studies (56%, trials 1, 2, 5, 8 and 9) reported sample size calculations adequately. Two of these (trials 2 and 9) did not recruit their target numbers of cases. Four studies (trials 3, 4, 6 and 7) did not report how sample sizes had been calculated. All five of the studies that quoted an expected CFR in controls over-estimated the CFR that was actually observed.

Recalculated sample sizes.  Minimum sample sizes were re-calculated to show a 25% and 50% reduction in mortality using the author's expected CFR among controls and assuming 80% power and 95% confidence. Trials 1, 2, 5 and 8 assumed relative effect reductions of 84%, 80%, 50% and 30%, and calculated sample sizes of 100, 100, 320 and 580, respectively. The sample sizes for trials 1, 2 and 5 were adequate to show these effects if they existed. Trial 2 could not recruit the calculated number of patients for logistics reasons (38 instead of 100). Trial 8 did not have enough participants to show 30% relative reduction in mortality despite recruiting 624 children. Only trials 5 and 8 had adequate sample sizes to detect 50% effect size and no trial had enough participants to detect 25% relative reduction.

Discussion

  1. Top of page
  2. Summary
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References

A major caution with a retrospective review is that failure to report does not automatically mean failure to do (Altman & Bland 1995), but the published article is often the only accessible measure of the conduct of a trial. However, this review has confirmed earlier reports that omission of sample size calculation in published papers of clinical trials is relatively common (Junker 1998; Moher et al. 2001). Previous studies also suggest that such omissions are commoner in trials which either report negative findings or which did not have sufficient numbers of cases to detect small effects (Freiman et al. 1978). In addition, most of the trials in this review, and particularly those that had mortality as primary outcome, tended to overestimate the CFR in controls that would occur during the trial, thus under-powering the study to detect the minimum reduction in mortality that the study was intended to detect.

Seventy-five per cent of the reviewed trials had adequate concealment of allocation schedule, a value higher than the 31% and 14.3% reported previously in systematic reviews of RCT that were not restricted to any particular field (Schultz et al. 1995; Moher et al. 1998). Only 11% of the trials reviewed reported the method of generation of allocation sequence, agreeing with a corresponding estimate of 15% (Moher et al. 1998). This is potentially an important finding given evidence that failures of appropriate generation of allocation sequence in RCTs can have a profound (positive or negative) effect on the results (Schultz et al. 1995; Moher et al. 1998; Kunz & Oxman 1998). As in previous reviews of RCTs (Schultz et al. 1995), most of the studies in this review included all randomized participants in the analysis. Seven of the nine studies in this review reported that the studies were double blinded, and five of these stated how this was achieved. This was higher than previous reports that between 22% and 50% of RCTs reporting adequately on blinding (Gillman & Runyan 1984; Schultz et al. 1995).

The importance of RCTs in defining medical practice places a responsibility on researchers and funding agencies to ensure that studies meet minimum standards defined in the CONSORT statement. Adequate sample size is essential if RCTs are to provide useful evidence on the effectiveness of treatments, and this is particularly important in CM where the pathogenesis might be multi-factorial and individual treatments may only be effective in a proportion of cases (Lou et al. 2001; Taylor et al. 2004). Researchers proposing to run RCTs obviously have to work within the funds that are made available and within the number of cases that can be recruited; these constraints may lead investigators to over-estimate the mortality that will occur under trial conditions or to somewhat arbitrarily set a high minimum level of reduction in mortality that would be worth detecting.

Few studies that certified the selection criteria were located a possible indication of the limited amount of work currently going on in the area of adjunctive treatment in CM. Also most of the reviewed trials were planned and conducted before the CONSORT guidelines were released, thus limiting the ability of the investigators to apply them. However is it conceivable that the reports of previous trials, which did not meet CONSORT standard, may have limited the conduct of further trials due to ethical concerns.

This review emphasizes the need to conduct trials of sufficient quality or size to detect moderate but potentially worthwhile improvements in treatment. Rigorous planning and multi-centre trials may be the solution to small number of subjects that could be recruited from one centre.

Acknowledgements

  1. Top of page
  2. Summary
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References

This study was originally submitted to LSHTM as partial requirement for the award of MSc in Epidemiology. Dr Hugh Reyburn supervised the thesis. Medical Research Council Laboratories, The Gambia sponsored my MSc programme.

References

  1. Top of page
  2. Summary
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References
  • Altman DG & Bland JM (1995) Absence of evidence is not evidence of absence. British Medical Journal 311, 485.
  • Altman DG, Schultz KF, Moher D et al. (2001) The Revised CONSORT statement for reporting randomised trials: explanation and elaboration. Annals of Internal Medicine 134, 663694.
  • Crawley J. Warruiru C, Mithwani S et al. (2000) Effect of phenobarbital on seizures frequency and mortality in childhood cerebral malaria: a randomised, controlled intervention study. Lancet 355, 710716.
  • Di Perri G, Di Perri IG, Monteiro GB et al. (1995) Pentoxifyllin as a supportive agent in the treatment of cerebral malaria in children. Journal of Infectious Diseases 171, 13171322.
  • Freiman JA, Chalmers TC, Smith H et al. (1978) The importance of beta, the type II error, and sample size in the design and interpretation of the randomised controlled trial: survey of 71 negative trials. New England Journal of Medicine 299, 690694.
  • Gillman MW & Runyan DK (1984) Bias in treatment assignment in controlled clinical trials. New England Journal of Medicine 310, 16101611.
  • Gordeuk V, Thuma P, Brittenham G et al. (1992) Effect of iron therapy on recovery from deep coma in children with cerebral malaria. New England Journal of Medicine 327, 14731477.
  • Hoffman SL, Rustama D, Punjabi NH et al. (1988) High dose dexamethasone in quinine-treated patients with cerebral malaria: a double blind placebo control trial. Journal of Infectious Diseases 158, 325331.
  • Hoffman SL (1996) Artemether in severe malaria-still too many deaths. New England Journal of Medicine 335, 124126.
  • Junker CA (1998) Adherence to published standard of reporting: a comparison of placebo controlled trials published in English and German. Journal of the American Medical Association 280, 247249.
  • Kochar DI, Kumawat BI & Kochar SK (1997) Seizures in cerebral malaria. Quarterly Journal of Medicine 90, 601607.
  • Kunz R & Oxman DA (1998) The unpredictability paradox: review of empirical comparisons of randomised and non-randomised clinical trials. British Medical Journal 317, 11851190.
  • Lou J, Lucas R & Grau GE (2001) Pathogenesis of cerebral malaria: recent experimental data and possible application for humans. Clinical Microbiological Review 14, 810820.
  • Moher D, Dulberg CS & Wells GA (1994) Statistical power, sample size, and their reporting in randomised controlled trials. Journal of the American Medical Association 272, 122124.
  • Moher D, Jadad AR, Nichol G et al. (1995) Assessing the quality of randomised controlled trials: an annotated bibliography of scales and checklists. Controlled Clinical Trials 16, 6273.
  • Moher D, Pham B, Jones A, et al. (1998) Does quality of report of randomised trials affect estimates of intervention efficacy reported in meta-analysis? Lancet 352, 609613.
  • Moher D, Schultz KF, Altman DG, for the CONSORT Group (2001) The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials. Lancet 357, 11911194.
  • Mulrow CD (1994) Systematic reviews: rationale for systematic reviews. British Medical Journal 309, 597599.
  • Schultz KF, Chalmers I, Hayes RJ et al. (1995) Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials. Journal of American Medical Association 273, 408412.
  • Taylor TE, Fu WJ, Carr RA et al. (2004) Differentiating the pathologies of cerebral malaria by postmortem parasite counts. Nature Medicine, 10, 143145.
  • Thuma PE, Mabeza GF, Biemba G et al. (1998) Effect of iron chelating therapy on mortality in Zambia children with cerebral malaria. Transactions of the Royal Society of Tropical Medicine and Hygiene 92, 214218.
  • Van Hensbroek MB, Palmer A, Onyiorah E et al. (1996) The effect of a monoclonal antibody to tumor necrosis factor on survival from childhood cerebral malaria. Journal of Infectious Diseases 174, 10911097.
  • Warrell DA, Looareesuwan S, Warrell MJ et al. (1982) Dexamethosaone proves deleterious in cerebral malaria: a double blind trial in 100 comatose patients. New England Journal of Medicine 309, 313319.
  • White NJ, Looareesuwan S, Phillips RE et al. (1988) Single dose phenobarbitone prevents convulsions in cerebral malaria. Lancet 2, 6466.
  • World Health Organisation (2000)Severe falciparum malaria. Transactions of the Royal Society of Tropical Medicine and Hygiene 94 (Suppl. 1), 190.