Please note that this comment, and the subsequent reply from the reviewer, was originally about the first version of this review (Pharmacology in acute spinal cord injury). The review has subsequently been revised to the present version (Steroids for acute spinal cord injury).
Summary of comments and criticisms.
The author of the criticism refers to the papers by Coleman et al 2000, and Hurlbert RJ which disagree with the conclusions of this review. He would like the following points addressed (each comment has a number with a corresponding response from the reviewers in the reply section below):
1. "NASCIS II" implied that there was a positive result in the primary efficacy analysis for the entire 487 patient sample. However, this analysis was in fact negative. A positive result was only found in a secondary analysis of a small subgroup (62 + 67 patients) splitting the sample before and after 8 hours.
2. The placebo group treated before 8 hours did poorly, not only when compared with the methylprednisolone group treated before 8 hours, but even when compared with the placebo group treated after 8 hours. Thus the positive result may have been caused by a weakness in the control group rather than any strength of methylprednisolone.
3. Most of the combined improvement from all patients in the subgroup (62 + 67 patients) was due to differences in the changes in the patients with incomplete lesions. This comparison involved only 22 patients in the methylprednisolone group and 24 patients in the placebo group.
4. The NASCIS II and III reports embody specific choices of statistical methods that have strongly shaped the reporting of results but have not been adequately challenged or even explained.
5. In NASCIS III, a randomization imbalance occurred that allocated a disproportionate number of patients with no motor deficit (and therefore no chance for recovery) to the lower dose control group. When this imbalance is controlled for, much of the superiority of the higher dose group seems to disappear.
6. Perhaps one half of the NASCIS III sample may have had at most a minor deficit. Thus, we do not know whether the results of these studies reflect the severely injured population to which they have been applied.
7. The numbers, tables, and figures in the published reports are scant and are inconsistently defined, making it impossible even for professional statisticians to duplicate the analyses, to guess the effect of changes in assumptions, or to supply the missing parts of the picture.
8. Nonetheless, even 9 years after NASCIS II, the primary data have not been made public.
9. The reporting of the NASCIS studies has fallen short of the guidelines of the ICH/FDA, and of the Evidence-based Medicine Group.
10. Despite the lucrative "off label" markets for methylprednisolone in Spinal Cord Injury, no Food and Drug Association indication has been obtained, and there has been no public process of validation.
11. These shortcomings have denied physicians the chance to use confidently a drug that many were enthusiastic about and have left them in an intolerably ambiguous position in their therapeutic choices, in their legal exposure, and in their ability to perform further research to help their patients.
12. Animal studies of the effect of Methylprednisolone and the human studies are different, and little work has been done to relate them explicitly. It is simply not true that the NASCIS studies either strongly confirm or are strongly confirmed by the animal studies.
In conclusion the use of methylprednisolone administration in the treatment of acute SCI is not proven as a standard of care, nor can it be considered a recommended treatment. Evidence of the drug's efficacy and impact is weak and may only represent random events. In the strictest sense, 24-hour administration of methylprednisolone must still be considered experimental for use in clinical SCI. Forty-eight-hour therapy is not recommended. These conclusions are important to consider in the design of future trials and in the medico-legal arena.
Bracken MB, Shepard MJ, Collins WF, Holford TR, Young W, Baskin DS et al. A randomized, controlled trial of methylprednisolone or naloxone in the treatment of acute spinal-cord injury. Results of the Second National Acute Spinal Cord Injury Study. N Engl J Med. 1990 May 17;322(20):1405-11.
Bracken MB, Shepard MJ, Holford TR, et al Administration of Methylprednisolone for 24 or 48 hours or Tirilazad Mesylate for 48 hours in the treatment of acute spinal cord injury. Results of the third national acute spinal cord injury randomized controlled trial. JAMA 1997;277:1597-1604.
Coleman WP, Benzel D, Cahill DW, Ducker T, Geisler F, Green B et al. A critical appraisal of the reporting of the National Acute Spinal Cord Injury Studies (II and III) of methylprednisolone in acute spinal cord injury. J Spinal Disord. 2000 Jun;13(3):185-99.
Hurlbert RJ. Methylprednisolone for acute spinal cord injury: an inappropriate standard of care. J Neurosurg 2000 Jul;93(1 Suppl):1-7.
Detailed responses to the comments reflected in the Criticism have been published elsewhere (1,2) and should be consulted by the interested reader.
1. The primary NASCIS 2 report (3) clearly stated that no benefit of methylprednisolone (MP) was observed in the total study group. In the a priori analysis of patients treated relatively quickly after injury (within 8 hours which was the modal time from injury to initiating therapy, and the only dichotomy analysed) patients treated with MP recovered significantly better than placebo treated patients. Examination of drug effect as a function of time to injury was a major hypothesis in the design of both NASCIS 2 and 3.
2. The comparison of placebo treated patients before versus after eight hours is not a randomized comparison and there is no reason to expect that these patients would be similar. The time taken to initiate therapy was largely a function of how quickly patients were admitted to hospital and there are many reasons why this may vary by severity of injury. The only valid comparisons for analysis are the ones reported, ie. comparisons of treatment (which was randomized) within the early and late time periods.
3. Statistically significant improvement in MP treated patients was observed and reported in both neurologically complete and incomplete patients as assessed in the emergency department.
4. The statistical procedure used to analyze NASCIS 2 and 3 was primarily analysis of covariance which is a standard form of analysis for randomized controlled trials. This methodology is described in any standard text.
5. In NASCIS 3 an imbalance at randomization was reported (4, table 2) which allocated somewhat more severely injured patients to Tiralazad mesylate. There was also a non-significant baseline difference in the two MP groups. Baseline neurological function was controlled in all statistical analyses and, as expected, the multivariate analysis of the two MP groups showed reduced improvement differences when the baseline differences were taken into account. These "controlled" analyses form the primary published results.
6. The NASCIS 3 report (4) shows severity of injury of all patients in the trial. Overall, for motor function 35.2% were quadriplegic; 31.0% paraplegic; 13.4% quadriparetic; 4.0% paraparetic and 14.4% normal although all normal motor responses had some sensory loss. After accounting for trial exclusion criteria (gunshot wounds, etc), the study population reflects the pattern of spinal injury seen in hospital emergency departments. Both NASCIS 2 and 3 showed efficacy of MP in severely injured patients, defined as having complete neurological loss below the level of injury.
7. Professional biostatisticians are among the NASCIS investigators and authors, were part of the review process at NEJM and JAMA, and sat on NIH panels overseeing the trials. Standard statistical procedures were used (item 4) and the neurological and functional definitions used are standard criteria promulgated by the American Spinal Injury Association, endorsed by the International Medical Society of Paraplegia, and widely adopted for clinical and research purposes around the world.
8. NASCIS data sets are available to recognized authoritative agencies and groups who submit a proposal describing their intended use of the data and demonstrate that they have the technical, biostatistical and clinical expertise to understand and analyse these complex data sets in an unbiased manner. Since NASCIS investigators continue to be funded by NIH for analyses of NASCIS 2 and 3, there is concern that analyses not be done which pre-empt publication of the same analyses by the initial investigators.
9. The ICH/FDA guidelines were published in 1996 but they enshrined principles and practices that have been evolving for many years. The NASCIS reports, even early ones, clearly meet both the spirit and intent of the recommendations.
10. The NASCIS studies are funded by the United States National Institute of Neurological Disease and Stroke. However, responsibility for seeking an indication for use in spinal injury from national drug regulatory agencies rests with the pharmaceutical company manufacturing the compound, Pharmacia-Upjohn Inc. NASCIS data is available for purposes of seeking regulatory approval of MP in any country. To the best of our knowledge, FDA approval has not been sought but an indication has been sought and obtained in a large number of other countries.
11. Physicians in many countries confidently use MP for spinal cord injury and have done so since 1990. The NASCIS 2 data supporting use has not changed since 1990. Nothing from the NASCIS studies prevents further research in spinal cord injury just as therapeutic discoveries in other areas of medicine do not stop research either. If MP has no benefit, comparing therapies to it should not pose a problem in demonstrating a new drug's superiority. If MP does confer benefit, comparison with it is necessary.
12. Animal studies serve two roles in developing scientific evidence. They prompt testing of therapies in humans after successful trial in animals and they provide biologic plausibility to the human evidence once it has been gathered. The weight of evidence from cat and other models using MP, which led to the initial trials, is strongly supportive of the role of MP (5). New experimental studies of MP in enhancing neuro-regeneration and playing other beneficial roles at the molecular level (6-8) provide further additional evidence of plausibility to support the human trials. This is an extraordinarily difficult but critically important area of human research and it is cause for concern that more trials of MP and other therapies are not being conducted. Currently, primary evidence of efficacy and safety from three trials, and secondary evidence from trials of related clinical conditions and animal studies, as reported in this Cochrane Review, support use of MP in the management of spinal cord injury. There is no other pharmacologic therapy with sufficient evidence to support use at this time.
1. Bracken MB, Aldrich EF, Herr DL et al. Clinical measurement, statistical analysis and risk benefit: controversies from trials of spinal injury. J Trauma 2000; 48:558-61.
2. Bracken MB. Methylprednisolone and spinal cord injury. J Neurosurg Spine 2000; 93:175-8.
3. Bracken MB, Shepard MJ, Collins WF et al. A randomized controlled trial of methylprednisolone or naloxone in the treatment of acute spinal cord injury: results of the second national acute spinal cord injury study. New Engl J Med 1990; 322:1405-11.
4. Bracken MB, Shepard MJ, Holford TR et al. Methylprednisolone administered for 24 or 48 hours, or 48 hour tirilazad mesylate, in the treatment of acute spinal cord injury; results of the third national acute spinal cord injury randomized controlled trial. JAMA 1997; 277:1597-1604.
5. Hall ED. The neuroprotective pharmacology of methylprednisolone. J Neurosurg 1992; 76:13-22.
6. Oudega M, Vargas CA, Weber AB et al. Long-term effects of methylprednisolone following transection of adult rat spinal cord. Eur J Neurosci 1999; 11:2453-64.
7. Banik NL, Matzelle D, Terry E et al. A new mechanism of methylprednisolone and other corticoids action demonstrated in vitro: inhibition of a proteinase (calpain) prevents myelin and cytoskeletal protein degradation. Brain Res 1997; 748:205-10.
8. Xu J, Fan G, Chen S et al. Methylprednisolone inhibition of TNF-alpha expression and NF-KB activation after spinal cord injury.