What is the role of risk-adjusted funnel plots in the analysis of radical cystectomy volume–outcome relationships?

Authors


Erik K Mayer, Clinical Lecturer in Surgery, Department of Surgery and Cancer, 10th Floor QEQM Building, St Mary’s Hospital Campus, Imperial College London, London, W2 1NY, UK. e-mail: e.mayer@imperial.ac.uk

Abstract

Study Type – Therapy (outcomes research)

Level of Evidence 2b

What’s known on the subject? and What does the study add?

The use of funnel plots has helped to overcome the limitations and risks of ranking surgical performance. The case for a more widespread application of funnel plots in assessing and reporting performance in surgery has been made. No study has previously published a funnel plot analysis of outcomes for radical cystectomy in England.

No Trust, using the final complex model for risk-adjustment, can be confidently said to have a performance worse than the national average for both mortality and re-intervention rates following radical cystectomy. Funnel plots act as a complementary method of validating volume-outcome data by displaying disaggregated outcomes at a provider level and reduce the opportunity for spurious labelling of outliers.

OBJECTIVE

• To explore whether risk-adjusted funnel plots are a useful adjunct to analyse volume–outcome data and to further facilitate our understanding of institutional performance data by combining funnel-plot methodology with an incremental statistical modelling approach.

PATIENTS AND METHODS

• Risk-adjusted funnel plots were generated for mortality and re-intervention rates after elective radical cystectomy using administrative data from NHS Hospital Trusts between 2000/01 and 2006/07. Trusts were divided into volume tertiles based on their average annual cystectomy rate.

• A funnel plot was produced for each of the following four incremental statistical models: model one (no adjustment), model two (adjusted for patient case mix variables), model three (case mix and ‘clustering’ of patients) and model four (additional adjustment for institutional structural and process-of-care variables).

RESULTS

• In the final complex model (model four), no Trusts had abnormally high mortality or re-intervention rates.

• Comparison of the funnel plots showed the importance of adjusting for certain confounding factors, such as the surgeon, at the institutional level, before they could be labelled as having truly outlying performance.

CONCLUSION

• Risk-adjusted funnel plots have a useful role to play as a component of a methodological framework for investigating the volume–outcome relationship at the institutional level. They can act as a complementary method of validating data by displaying disaggregated outcomes at provider level and account for unmeasured confounders, so reducing the opportunity for spurious labelling of outliers.

Ancillary