SEARCH

SEARCH BY CITATION

Keywords:

  • clinical trials;
  • confidence intervals;
  • data analysis and statistics;
  • evidence-based medicine
Abbreviations
SR

systematic review

RR

risk ratio.

CASE SCENARIO

  1. Top of page
  2. CASE SCENARIO
  3. CLINICAL QUESTION
  4. FINDING THE BEST EVIDENCE
  5. THE EVIDENCE
  6. UNDERSTANDING P-VALUES AND CIs
  7. APPLYING CIs AND P-VALUES TO CLINICAL PRACTICE
  8. APPLYING THE RESULTS TO PATIENT CARE
  9. CONCLUSION
  10. CONFLICT OF INTEREST
  11. REFERENCES

A family practice doctor from a neighbouring office stops you in the hall to ask your opinion. Her elderly mother has developed dysuria, urinary frequency and suprapubic pain. A formal urine analysis and a urine culture are confirmatory of a UTI. She does not have a history of previous UTIs and her only other medical history is mild dementia. Your colleague wants your advice on whether she can treat her mother with a single dose of antibiotics given that she has no fever, no flank tenderness, and has a normal white blood cell count. To answer this colleague’s question with the most current, best evidence available you decide to perform a focused literature search [1].

CLINICAL QUESTION

  1. Top of page
  2. CASE SCENARIO
  3. CLINICAL QUESTION
  4. FINDING THE BEST EVIDENCE
  5. THE EVIDENCE
  6. UNDERSTANDING P-VALUES AND CIs
  7. APPLYING CIs AND P-VALUES TO CLINICAL PRACTICE
  8. APPLYING THE RESULTS TO PATIENT CARE
  9. CONCLUSION
  10. CONFLICT OF INTEREST
  11. REFERENCES

In an elderly woman with an uncomplicated symptomatic UTI (population), how does treatment with a single dose of antibiotics (interventions) compare to a short course of antibiotics (comparator) in terms of preventing recurrent UTIs (outcome)?

FINDING THE BEST EVIDENCE

  1. Top of page
  2. CASE SCENARIO
  3. CLINICAL QUESTION
  4. FINDING THE BEST EVIDENCE
  5. THE EVIDENCE
  6. UNDERSTANDING P-VALUES AND CIs
  7. APPLYING CIs AND P-VALUES TO CLINICAL PRACTICE
  8. APPLYING THE RESULTS TO PATIENT CARE
  9. CONCLUSION
  10. CONFLICT OF INTEREST
  11. REFERENCES

Recognising this as a question relating to therapy that would be ideally answered by a systematic review (SR) and meta-analysis, you search the Cochrane Database of SRs using the search terms ‘antibiotics’, ‘elderly’, and ‘UTI’. The first article from a list of five hits (search date: 12/5/2009) is a recent SR of the literature by Lutters et al. [2] that compares the benefits and harms of a single dose vs short course (3–6 days) of antibiotics. You determine that this SR with the title, ‘Antibiotic duration for treating uncomplicated, symptomatic lower urinary tract infections in elderly women’, by Lutters et al. [2] appears to address your clinical question. Representing a study from the Cochrane group, which is known for its systematically rigorous SRs, you place a high degree of confidence into its results [3].

THE EVIDENCE

  1. Top of page
  2. CASE SCENARIO
  3. CLINICAL QUESTION
  4. FINDING THE BEST EVIDENCE
  5. THE EVIDENCE
  6. UNDERSTANDING P-VALUES AND CIs
  7. APPLYING CIs AND P-VALUES TO CLINICAL PRACTICE
  8. APPLYING THE RESULTS TO PATIENT CARE
  9. CONCLUSION
  10. CONFLICT OF INTEREST
  11. REFERENCES

Reviewing the study results, the main finding of the study was that a recurrent UTI in the short term (<2 weeks after treatment) was more common in those given single dose vs a short course (risk ratio (RR) 2.0, 95% CI 1.1–3.8; P= 0.034). However, at long-term follow-up, the incidence of UTI was comparable between groups (RR 1.18, 95% CI 0.59–2.32). Furthermore, the study found no difference between single dose and short course therapy for the rates of adverse reaction, re-infection rates, and treatment acceptability by patients. You realise that to better understand these results and their clinical relevance it is important to understand two closely related statistical measures being reported: P-values and CIs.

UNDERSTANDING P-VALUES AND CIs

  1. Top of page
  2. CASE SCENARIO
  3. CLINICAL QUESTION
  4. FINDING THE BEST EVIDENCE
  5. THE EVIDENCE
  6. UNDERSTANDING P-VALUES AND CIs
  7. APPLYING CIs AND P-VALUES TO CLINICAL PRACTICE
  8. APPLYING THE RESULTS TO PATIENT CARE
  9. CONCLUSION
  10. CONFLICT OF INTEREST
  11. REFERENCES

P-values and CIs refer to two different, yet complementary approaches to statistical analysis: hypothesis-testing and estimation [4]. In the hypothesis-testing framework, the P value describes the probability that the observed result (or a larger difference) is due to chance if, in fact, there is no true difference between the groups (i.e. the ‘null hypothesis’ of no effect is true, despite the observed difference). A very small P value suggests that the null hypothesis is highly unlikely to be true, and thus suggests that the observed effect is real. By convention, a P < 0.05 is considered ‘statistically significant’ and is widely considered adequate proof of an effect, although this assumption is often misguided [4].

In contrast, the estimation approach seeks to quantify the effect of interest and provide a measure of the associated degree of uncertainty by means of a CI. The CI represents a range of values on either side of the estimate between which we can be 95% sure that the true values falls. A CI therefore communicates the degree of uncertainty of the study result as an estimate of the ‘true’ population value. In contrast, a P-value only measures the probability of the observed results being a chance phenomenon [5]. P-values tell us nothing about the size of a difference or even the direction of the difference and are therefore uninformative when provided in isolation.

APPLYING CIs AND P-VALUES TO CLINICAL PRACTICE

  1. Top of page
  2. CASE SCENARIO
  3. CLINICAL QUESTION
  4. FINDING THE BEST EVIDENCE
  5. THE EVIDENCE
  6. UNDERSTANDING P-VALUES AND CIs
  7. APPLYING CIs AND P-VALUES TO CLINICAL PRACTICE
  8. APPLYING THE RESULTS TO PATIENT CARE
  9. CONCLUSION
  10. CONFLICT OF INTEREST
  11. REFERENCES

The SR by Lutters et al. [2] found that the risk of a recurrent UTI in elderly women treated with a single dose of antibiotics was significantly greater than those treated with a short course of antibiotics. In ‘plain’ language, the RR of 2.01 indicates that women treated with a single dose of antibiotics were approximately twice as likely to have a recurrent UTI as those treated with a short course (3–6 days). However, the CI around this point estimate reflects considerable uncertainty. The relative risk of a recurrent UTI may actually be only 1.05 times higher or as much as 3.84 times higher. Based on numbers from this SR, a short-term course reduced the absolute rates of UTIs from 22.1% (38/172) to 10.3% (19/184), which corresponded to an absolute risk reduction of 11.8% or a number needed-to-treat of nine (95% CI 5–24). In other words, as few as five women, or as many as 24 women, would need to be treated with a short course of antibiotics (instead of a single dose regimen) to prevent a recurrent UTI.

In terms of preventing recurrent UTIs long-term, this SR found no significantly increased risk in women treated with a single dose vs a short course of antibiotics (RR 1.18, 95% CI 0.95–2.32; P= 0.64). The lack of statistical significance is reflected in the P value ≥0.05 as well as the 95% CI crossing 1, the value of no difference. You also note that the risk of a recurrent UTI in the single-dose group was ≈40%.

A common misinterpretation of negative studies such as these is that the intervention of interest (i.e. a short course of antibiotics) is indeed not more effective when compared with the control treatment (i.e. a single dose of antibiotics). This conclusion is flawed without review of the CIs because the study may have been underpowered to show a significant effect [6]. In this case, the CI includes the possibility that a single dose of antibiotics reduces the risk of recurrent UTIs long-term by 5% as well as the possibility that it is associated with a 132% increased risk of recurrent UTIs, as compared with short-course treatment. Therefore, although we cannot exclude the possibility that a short-term course may in fact do more harm than good (that is, increase your risk of recurrent UTIs long-term) or only has the same effectiveness as a single dose, the CI is also consistent with a potential large benefit. It would therefore be wrong to conclude that this SR proves that a short-term course of antibiotics is not more effective than a single dose of antibiotics. Although this conclusion may be correct, it also appears quite possible that the study was simply underpowered, i.e. had too small a sample size with too few events to draw a firm conclusion. In cases such as these, further studies with a larger number of patients may provide the power to prove a statistically significant effect. Table 1 shows the relationship of study size and the corresponding CI of the effect size.

Table 1.  Series of hypothetical randomized controlled trials with the same effect sizes but increasing sample sizes showing the effect on the associated CIs*
Treatment/event rateControl/event rateRR reduction (RRR), %Calculated 95% CI around the RRR, %
1/42/450−255.0 to 93.0
5/2010/2050 −20.1 to 79.2
10/4020/4050   7.1 to 73.1
25/10050/10050  26.0 to 66.2
250/1000500/100050  43.4 to 55.8

APPLYING THE RESULTS TO PATIENT CARE

  1. Top of page
  2. CASE SCENARIO
  3. CLINICAL QUESTION
  4. FINDING THE BEST EVIDENCE
  5. THE EVIDENCE
  6. UNDERSTANDING P-VALUES AND CIs
  7. APPLYING CIs AND P-VALUES TO CLINICAL PRACTICE
  8. APPLYING THE RESULTS TO PATIENT CARE
  9. CONCLUSION
  10. CONFLICT OF INTEREST
  11. REFERENCES

The findings of the SR by Lutters et al. [2] provide evidence that a short course of antibiotics is more effective than single-dose treatment for the prevention of recurrent UTIs short-term. Although this SR failed to prove the benefit of a short course of antibiotics over a single dose long-term, its results did not exclude a potential benefit. Based on the current best evidence and having further determined that the results of the study appear applicable to your colleague’s mother, you recommend a short course of antibiotics of 3–6 days in duration.

CONCLUSION

  1. Top of page
  2. CASE SCENARIO
  3. CLINICAL QUESTION
  4. FINDING THE BEST EVIDENCE
  5. THE EVIDENCE
  6. UNDERSTANDING P-VALUES AND CIs
  7. APPLYING CIs AND P-VALUES TO CLINICAL PRACTICE
  8. APPLYING THE RESULTS TO PATIENT CARE
  9. CONCLUSION
  10. CONFLICT OF INTEREST
  11. REFERENCES

CIs provide a measure of precision that characterizes the range of values in which we can expect the ‘true’ effect size to lie with a given degree of certainty. The appropriate interpretation of CIs is equally important in studies with positive and negative results and therefore an important aspect of evidence-based clinical practice of urology.

REFERENCES

  1. Top of page
  2. CASE SCENARIO
  3. CLINICAL QUESTION
  4. FINDING THE BEST EVIDENCE
  5. THE EVIDENCE
  6. UNDERSTANDING P-VALUES AND CIs
  7. APPLYING CIs AND P-VALUES TO CLINICAL PRACTICE
  8. APPLYING THE RESULTS TO PATIENT CARE
  9. CONCLUSION
  10. CONFLICT OF INTEREST
  11. REFERENCES