Dear Editor

The authors thank Dr McKenzie for his thoughtful consideration of the study. Most of his points revolve around a recent publication by Drs Conzemius and Evans describing the placebo effects in the caregiver assessment of lameness. The important things to consider when comparing these 2 studies are:

  1. The caregiver assessment used in the Conzemius study was specifically for evaluating lameness only. There is clearly no argument that, if what one wants to measure is lameness, force plate gait analysis is more sensitive than the eye of an owner, or a veterinarian for that matter. The Canine BPI was designed to quantify chronic pain, not lameness specifically. It takes into account a wide variety of behaviors and lameness is not actually even addressed as a specific behavior in the instrument. Lameness may be one of the behaviors that the owner integrates into their assessment of their dogs' level of discomfort, but as our study shows, it is not the primary one.
  2. Very importantly, the lameness scoring system that the owners were instructed to use in the Conzemius study has not been validated, to the best of our knowledge. There are no references for the development of this lameness scoring system or independent evaluations of validity or reliability testing. I would venture to say that the lameness scoring system would not stand up well to such testing as there are a multitude of behaviors listed within every category of lameness and, most importantly, the categories of lameness are not in any way mutually exclusive. This ambiguity and overlap in the category descriptions will lead to a high level of variability in the owner responses, making it not a particularly powerful outcome assessment instrument, as was suggested in the study.

So comparing an unvalidated instrument designed to measure lameness to the Canine Brief Pain Inventory, which has been validated to assess chronic pain, is like comparing apples and oranges.

With regard to the general comments about subjective versus objective outcome measures, the important things to consider are:

  1. Placebo effects occur in both objective and subjective measures. Placebo effects generally mean that a positive effect is documented in spite of the fact that there is no intervention. As described in our paper, regression to the mean is a classic placebo effect that is the likely culprit for improvement documented using objective measures such as force plate gait analysis in animals randomized to the placebo arm of a study.
  2. Subjective measures are absolutely prone to greater placebo effects than objective measures, but that does not mean that the subjective measure is any less useful or valid. If a subjective measure is appropriately developed, it can be an extremely valuable, valid, and reliable outcome assessment instrument.
  3. In all studies, the use of a control group is pivotal to understanding the true impact of the intervention. In uncontrolled studies, the positive effect of an intervention is overestimated. This is due to the fact that part of the positive effect is placebo (ie, not actually due to the intervention). How much is placebo versus true intervention effect cannot be estimated without a control arm in the study regardless of whether the outcome measure is subjective or objective. Because placebo effects can be greater in subjective measures the importance of a control is even greater when a subjective measure is used.

In the end, the most important thing to consider when choosing an outcome assessment instrument for a study is choosing a validated tool. The next step is choosing from amongst the validated tools the one that will allow the investigator to answer the question they are most interested in asking. If one wants to know whether an intervention decreases lameness, force plate gait analysis is an excellent choice. If one wants to know whether an intervention improves chronic pain behaviors during a dog's daily routine in its home environment, then the Canine BPI is a well validated option.