Teaching critical appraisal skills in health care settings

 

Abstract

Background

Critical appraisal is the process of assessing and interpreting evidence by systematically considering its validity, results and relevance to an individual's work. Within the last decade critical appraisal has been added as a topic to many medical school and UK Royal College curricula, and several continuing professional development ventures have been funded to provide further training.

Objectives

To assess the effects of teaching critical appraisal skills to health professionals, on the process of care, patient outcomes and knowledge of health professionals.

Search strategy

We searched The Cochrane Library (to Issue 2 2000), MEDLINE (1966 to 1997), EMBASE (1980 to 1997), Eric (1966 to 1997), Cinahl (1982 to 1997), Lisa (1976 to 1997), Sigle (1980 to 1997), Science Citation Index (1981 to 1997), PsycLit (1974 to 1997), the world-wide-web, and reference lists of articles. We also contacted major medical education centres.

Selection criteria

Randomised trials, controlled clinical trials, controlled before and after studies and interrupted time series analyses of educational interventions teaching critical appraisal to health professionals. The outcomes were: process of care; patient mortality, quality of life, and satisfaction; and health professional knowledge/awareness based upon objective, standardised, validated instruments.

Data collection and analysis

Two reviewers independently extracted data and three reviewers independently assessed study quality.

Main results

One USA hospital-based randomised trial was included involving 44 doctors. The outcome assessed was critical appraisal knowledge. Process of care, patient health or attitude/awareness outcomes were not assessed. Critical appraisal teaching was reported to have resulted in a 25% improvement (adjusted figure) in critical appraisal knowledge in the intervention group compared to a 6% improvement in the control group, which was statistically significant (p=0.02).

Authors' conclusions

There is evidence that critical appraisal teaching has positive effects on participants' knowledge, but as only one study met the inclusion criteria the validity of drawing general conclusions about the effects of teaching critical appraisal is debatable. There are large gaps in the evidence as to whether teaching critical appraisal impacts on decision-making or patient outcomes. It is also unclear whether the size of benefit seen is large enough to be of practical significance, or whether this varies according to participant background or teaching method. The evidence supporting all outcomes is weakened by the generally poorly designed, executed and reported studies that we found.

 

Plain language summary

Teaching critical appraisal skills to health professionals improves knowledge but there is lack of evidence that it changes the process of care or patient outcomes

Critical appraisal involves interpreting information in a systematic and objective manner. This review looked at whether teaching critical appraisal skills to health professionals led to changes in the process of care, patient outcomes or health professionals' knowledge/awareness. The review found that teaching critical appraisal skills to health professionals improved their knowledge of these skills. However there was a lack of good quality evidence as to whether teaching critical appraisal skills led to changes in the process of care or to changes in patient outcomes.