The editorial by Kleinig et al. highlights the importance of a deeper level of understanding of the science and research methodology of the literature on this subject. Many of the specific criticisms of the original editorials were addressed by the accompanying letters in the same issue. The authors have neither commented on the poor inter-rater reliability of the mRS nor on the revelations by Shy on the altered ECASS III results. Rather than restate an already published rebuttal, I will simply highlight several assertions made by the authors.
- ‘In all trials of alteplase-treated patients within 3 h of onset, regardless of age, the NNT for “cure” is 11.’ This is based on the Wardlaw meta-analysis, which is based on the pooled analysis. Newman has already highlighted the immense clinical and statistical heterogeneity of the studies, noting that these data must be clinically and statistically homogenous to be validly pooled. While the 2009 Cochrane review reports no statistically significant difference between the drugs, the 2012 Wardlaw analysis reports only on tPA. There is neither a theoretical basis nor any clinical data from the MI studies that suggest tPA is more likely to demonstrate benefit than any other agent. Indeed, it is methodologically flawed to extract subgroups (post hoc) from different trials to combine in meta-analysis, as it removes any possibility of balancing confounders, and magically changes negative studies into a ‘positive’ result. Pooled analyses are not randomised trials and are unacceptable as proof of benefit. The kindest interpretation is that it is hypothesis generating. The use of the word ‘cure’ is disingenuous and not consistent with clinical experience and implies that stroke thrombolysis achieves better results than for AMI.
- ‘Shift analysis … suggests a number needed to functionally benefit from tPA of 3–4 in patients treated within 3 h.’ This is based on the Saver statistical study of 100 model patients drawn from the original NINDS study (n = 624). This is unreliable and deceptive. Readers are referred to the Radecki et al. review of IST-3 that highlights the problems of shift analysis. This includes reference to an earlier paper by the IST-3 authors ‘that an ordinal analysis was not universally applicable and that underlying assumptions necessary for validity would not be met by the IST-3 data set.’ The ‘favourable shift,’ a secondary outcome, is ‘an ambiguous concept of unclear clinical significance.’ Again, the kindest interpretation of this is hypothesis generating.
- ‘IST-3 fulfilled this wish [to replicate NINDS], showing clear evidence of benefit …’ This highlights the problem of having your cake and eating it too. Early in the editorial, the authors state that IST-3 is largely a trial in patients with relative contraindications to tPA and that it was a PROBE design and that the primary outcome was negative. This is not a replication of NINDS. In fact, in the early phase of IST-3 that was both NINDS-like and measured outcomes in person, and blinded to treatment arm, all trends favoured placebo. As a result, it is possible that the NNT approaches infinity.
The authors conclude by stating the often promoted future of acute stroke research: using advanced neuroimaging to refine selection criteria. However, three recent publications in The New England Journal of Medicine demonstrated failures of these hypotheses, leading one author to state ‘the imaging selection hypothesis is flawed as conceived’. We need to be aware of all these issues when assessing the evidence. More importantly, emergency physicians would welcome the assistance of stroke physicians to replicate the science and work out who, if anyone, really benefits, as was done for AMI.