We have the same concerns we published in our earlier letter to the editor about Pouchelon and colleagues' study, and wish to refute certain statements made in their letter.1–3 Our point was not to indict retrospective studies per se. Rather, our objections specifically concern retrospective studies of treatment efficacy, which are biased by design. We agree that generating data is beneficial with the caveat that the data be derived from rationally designed studies that are analyzed appropriately.
Causal inference: Three studies have attempted to examine ACE inhibitors' ability to delay time from diagnosis to onset of heart failure in dogs with mitral regurgitation (MR) from myxomatous valve degeneration (MVD). The SVEP trial found no prolongation in this interval in Cavalier King Charles (CKC) Spaniels.4 The VETPROOF trial in other breeds (OB) concluded that “improvement in the primary endpoint, CHF-free survival, was not significant.”5 Pouchelon et al 3 stated, “No difference in the time to onset of CHF was found between the BNZ (benazepril) and UT (untreated) groups in the whole study population or in the OB and CKC populations”—a statement they failed to repeat in their conclusions or abstract. All three studies have now found ACE inhibitors do not prolong the time to onset of heart failure in any dogs with MVD.
A striking difference between the SVEP and VETPROOF trials and Pouchelon and colleagues' retrospective study is that the former were randomized, multicenter, blinded clinical trials. The greatest threat to validity in Pouchelon and colleagues arises from uncontrolled confounding due to clinical indication for treatment. The choice of treatments in a nonrandomized study is left entirely to the clinician's discretion, and it is impossible to make such a choice devoid of considering the very factors directly influencing adverse endpoints, namely, but not limited to, disease severity, perception of prognosis, owner economic considerations, and compliance. The near-certain presence of confounding by indication in Pouchelon and colleagues likely indicates that Type I errors occur with a higher than nominal frequency, confidence intervals are too narrow, and P-values are underestimated.6 Such uncontrolled confounding provides a compelling contraindication to performing statistical tests whose validity is predicated on the assumption of randomization of unmeasured or unmeasurable confounders among strata of controlled variables. Although Pouchelon and colleagues provide no documentation that their multivariate regression models control for any confounders, it would be extremely difficult in their retrospective study to circumvent the confounding by indication problem. There are numerous examples of observational (ergo nonrandomized) studies of treatment efficacy whose findings have been refuted by subsequent randomized clinical trials.7
Endpoints, all-cause death, and sudden death: We believe that it is generally ill advised to evaluate all-cause mortality in a study of cardiac treatment unless the treatment is known to affect noncardiac mortality and is prescribed for those causes of mortality. If cardiac mortality had been lessened by benazepril in Pouchelon and colleagues and all-cause mortality had increased, that finding would be interesting. Instead mortality from heart failure was not lessened and all-cause mortality improved, suggesting that benazepril had a beneficial effect unrelated to underlying cardiac disease. In the abstract, benazepril instead was represented as beneficial for dogs with cardiac disease: (“… median survival time to all causes of death in the BNZ group was significantly longer than in the UT group …” and so “BNZ had beneficial effects in asymptomatic dogs … affected by MVD with moderate-to-severe MR”). While we do not dispute that ACE inhibitors may have other beneficial effects, dogs that died of noncardiac causes succumbed to cancer, renal insufficiency, neurological diseases, surgery, diabetes mellitus, trauma, and advanced age. Benazepril might have benefited the 7 dogs with renal failure, but not the other 28 dogs. We are skeptical that the benefit in renal failure dogs was profound enough to produce a benefit to the whole study population. Thus, we believe the proper interpretation of the findings is one of chance (Type I error).
The authors conclude that administering benazepril prolongs the time until either heart failure or sudden death. Including time to sudden death with time to heart failure would be reasonable in a group of Doberman Pinschers with dilated cardiomyopathy where sudden death is common, but is unreasonable in small dogs with primary MR because sudden death here is relatively rare. This likely became significant only because all 5 dogs that died suddenly happened to be untreated.
Dog numbers: Of the 141 dogs, only 15 died of heart disease. Therefore, for cardiac-related mortality analyses, 89% were censored. It is surprising to us that the authors would make strong statements about treatment efficacy on survival based on such a small number of dogs succumbing to cardiac disease. Moreover, the findings are valid only insofar as censoring is unrelated to treatment group and patient outcome, an assumption unverifiable in the manuscript.
Pulmonary hypertension (PH) and disease severity: The authors contend that dogs with even severe PH were appropriately included because MR can cause severe PH, even in dogs with a normal-sized left atrium (LA). One study they cited examined severity of MR on color flow Doppler MR jet size in relation to LA size.8 The authors identified clinically unaffected dogs with MR and a normal LA size where the color flow jet filled a significant proportion of the LA and concluded moderate to severe MR existed. We believe that the premise of severe PH with a normal-sized LA is without basis given that we know of no mechanism by which the ventricle can eject more than 50% of regurgitant flow into the LA each systolic interval and have the LA remain normal in size.
In conclusion, the study presented by Pouchelon and colleagues was unfortunately flawed in too many ways to reach any valid conclusion on benazepril's effects in dogs with primary MR before the onset of heart failure.