Volume 24, Issue 8
Research Article

Funnel plots for comparing institutional performance

David J. Spiegelhalter

Corresponding Author

E-mail address: david.spiegelhalter@mrc‐bsu.cam.ac.uk

MRC Biostatistics Unit, Institute of Public Health, Cambridge CB2 2SR, U.K.

MRC Biostatistics Unit, Institute of Public Health, Robinson Way, Cambridge CB2 2SR, U.K.Search for more papers by this author
First published: 29 November 2004
Citations: 471

Abstract

‘Funnel plots’ are recommended as a graphical aid for institutional comparisons, in which an estimate of an underlying quantity is plotted against an interpretable measure of its precision. ‘Control limits’ form a funnel around the target outcome, in a close analogy to standard Shewhart control charts. Examples are given for comparing proportions and changes in rates, assessing association between outcome and volume of cases, and dealing with over‐dispersion due to unmeasured risk factors. We conclude that funnel plots are flexible, attractively simple, and avoid spurious ranking of institutions into ‘league tables’. Copyright © 2004 John Wiley & Sons, Ltd.

Number of times cited according to CrossRef: 471

  • Determinants of Variation in the Use of Adjuvant Chemotherapy for Stage III Colon Cancer in England, Clinical Oncology, 10.1016/j.clon.2019.12.008, (2020).
  • Multisociety expert consensus systems of care document 2019 AATS/ACC/SCAI/STS expert consensus systems of care document: Operator and institutional recommendations and requirements for transcatheter mitral valve intervention: A Joint Report of the American Association for Thoracic Surgery, the American College of Cardiology, the Society for Cardiovascular Angiography and Interventions, and The Society of Thoracic Surgeons, Catheterization and Cardiovascular Interventions, 10.1002/ccd.28671, 95, 5, (866-884), (2020).
  • Using publicly available flight data to analyze health disparities in aeromedical retrieval, Journal of the American College of Emergency Physicians Open, 10.1002/emp2.12121, 1, 4, (453-459), (2020).
  • Steering by their own lights: Why regulators across Europe use different indicators to measure healthcare quality, Health Policy, 10.1016/j.healthpol.2020.02.012, (2020).
  • Bayesian central statistical monitoring using finite mixture models in multicenter clinical trials, Contemporary Clinical Trials Communications, 10.1016/j.conctc.2020.100566, (100566), (2020).
  • Results in the ESPN/ERA-EDTA Registry suggest disparities in access to kidney transplantation but little variation in graft survival of children across Europe., Kidney International, 10.1016/j.kint.2020.03.029, (2020).
  • Effect of exercise prehabilitation on functional status of patients undergoing bowel resection: a systematic review, ANZ Journal of Surgery, 10.1111/ans.15659, 90, 5, (693-701), (2020).
  • Amyloid duration is associated with preclinical cognitive decline and tau PET, Alzheimer's & Dementia: Diagnosis, Assessment & Disease Monitoring, 10.1002/dad2.12007, 12, 1, (2020).
  • Long‐term outcomes of mitral valve endocarditis: improved survival through collaborative management, ANZ Journal of Surgery, 10.1111/ans.15814, 90, 5, (757-761), (2020).
  • Plasma pyroglutamate‐modified amyloid beta differentiates amyloid pathology, Alzheimer's & Dementia: Diagnosis, Assessment & Disease Monitoring, 10.1002/dad2.12029, 12, 1, (2020).
  • Pancreaticoduodenectomy outcomes in Australia, ANZ Journal of Surgery, 10.1111/ans.15778, 90, 5, (938-939), (2020).
  • Potentially avoidable urology mortality in Australia, ANZ Journal of Surgery, 10.1111/ans.15890, 90, 5, (653-653), (2020).
  • Down syndrome: Distribution of brain amyloid in mild cognitive impairment, Alzheimer's & Dementia: Diagnosis, Assessment & Disease Monitoring, 10.1002/dad2.12013, 12, 1, (2020).
  • Interaction between APOE genotype and diabetes in cognitive decline, Alzheimer's & Dementia: Diagnosis, Assessment & Disease Monitoring, 10.1002/dad2.12006, 12, 1, (2020).
  • Utility of the American College of Surgeons National Surgical Quality Improvement Program surgical risk calculator in predicting mortality in an Australian acute surgical unit, ANZ Journal of Surgery, 10.1111/ans.15892, 90, 5, (746-751), (2020).
  • National Surgical Quality Improvement Program risk calculator validity in South Australian laryngectomy patients, ANZ Journal of Surgery, 10.1111/ans.15807, 90, 5, (740-745), (2020).
  • Transanal total mesorectal excision: the new kid on the block or a false dawn?, ANZ Journal of Surgery, 10.1111/ans.15808, 90, 5, (651-652), (2020).
  • Commentary on Composite cognitive and functional measures for early stage Alzheimer's disease trials, Alzheimer's & Dementia: Diagnosis, Assessment & Disease Monitoring, 10.1002/dad2.12012, 12, 1, (2020).
  • Reducing variation in hospital mortality for alcohol‐related liver disease in North West England, Alimentary Pharmacology & Therapeutics, 10.1111/apt.15781, 52, 1, (182-195), (2020).
  • Transport of orally delivered dsRNA in southern green stink bug, Nezara viridula, Archives of Insect Biochemistry and Physiology, 10.1002/arch.21692, 104, 4, (2020).
  • The possible role of NF‐κB1 Rs28362491 polymorphism in male fertility of Egyptian population, Andrologia, 10.1111/and.13659, 52, 7, (2020).
  • Flower‐like shaped Bi12TiO20/g‐C3N4 heterojunction for effective elimination of organic pollutants: Preparation, characterization, and mechanism study, Applied Organometallic Chemistry, 10.1002/aoc.5702, 34, 8, (2020).
  • Ustekinumab is associated with superior effectiveness outcomes compared to vedolizumab in Crohn’s disease patients with prior failure to anti‐TNF treatment, Alimentary Pharmacology & Therapeutics, 10.1111/apt.15745, 52, 1, (123-134), (2020).
  • Comparative proteomic analysis of mitochondrial proteins from maize CMS‐C sterile, maintainer and restorer anthers, The Plant Genome, 10.1002/tpg2.20022, 13, 2, (2020).
  • Risk stratification for prostate cancer management: value of the Cambridge Prognostic Group classification for assessing treatment allocation, BMC Medicine, 10.1186/s12916-020-01588-9, 18, 1, (2020).
  • 2019 AATS/ACC/SCAI/STS expert consensus systems of care document: Operator and institutional recommendations and requirements for transcatheter mitral valve intervention, The Journal of Thoracic and Cardiovascular Surgery, 10.1016/j.jtcvs.2019.12.001, 160, 1, (72-92), (2020).
  • 2019 AATS/ACC/SCAI/STS Expert Consensus Systems of Care Document: Operator and Institutional Recommendations and Requirements for Transcatheter Mitral Valve Intervention, The Annals of Thoracic Surgery, 10.1016/j.athoracsur.2019.11.004, 110, 1, (316-335), (2020).
  • Risk‐adjusted hospital clinical management issue rates using data from the Victorian Audit of Surgical Mortality, ANZ Journal of Surgery, 10.1111/ans.15896, 90, 5, (728-733), (2020).
  • Incidence and survival of rare cancers in the US and Europe, Cancer Medicine, 10.1002/cam4.3137, 9, 15, (5632-5642), (2020).
  • Understanding the large heterogeneity in hospital readmissions and mortality for acute myocardial infarction, Health Policy, 10.1016/j.healthpol.2020.04.004, (2020).
  • Impact of COVID-19 pandemic on surgical neuro-oncology multi-disciplinary team decision making: a national survey (COVID-CNSMDT Study), BMJ Open, 10.1136/bmjopen-2020-040898, 10, 8, (e040898), (2020).
  • Closing the gap in implementation of HIV clinical guidelines in a low resource setting using electronic medical records, BMC Health Services Research, 10.1186/s12913-020-05613-8, 20, 1, (2020).
  • Commentary: Safety in numbers, The Journal of Thoracic and Cardiovascular Surgery, 10.1016/j.jtcvs.2020.07.058, (2020).
  • Digital transformation of the emergency department-inpatient interface (EDii): integration for future innovation, Australian Health Review, 10.1071/AH18176, 44, 5, (666), (2020).
  • Improving cardiac surgical quality: lessons from the Japanese experience, BMJ Quality & Safety, 10.1136/bmjqs-2019-010125, 29, 7, (531-535), (2020).
  • 2019 AATS/ACC/SCAI/STS Expert Consensus Systems of Care Document: Operator and Institutional Recommendations and Requirements for Transcatheter Mitral Valve Intervention, Journal of the American College of Cardiology, 10.1016/j.jacc.2019.12.002, 76, 1, (96-117), (2020).
  • Short- and Long-Term Outcome in Critically Ill Children After Acute Interhospital Transport to a PICU in Sweden*, Pediatric Critical Care Medicine, 10.1097/PCC.0000000000002319, 21, 7, (e414-e425), (2020).
  • Whole blood transfusion versus component therapy in trauma resuscitation: a systematic review and meta‐analysis, Journal of the American College of Emergency Physicians Open, 10.1002/emp2.12089, 1, 4, (633-641), (2020).
  • Rapid collapse of the inferior vena cava in a patient with cardiac arrest induced by anaphylactic shock after iodinated contrast medium injection, Journal of the American College of Emergency Physicians Open, 10.1002/emp2.12166, 1, 4, (379-382), (2020).
  • An investigation of clinical decisionmaking: identifying important factors in treatment planning for suicidal patients in the emergency department, Journal of the American College of Emergency Physicians Open, 10.1002/emp2.12087, 1, 4, (383-391), (2020).
  • Influence of Revision Surgeries and Revision Rate on Quality Indicators in Certified Centres for Joint Replacement (EndoProthetikZentren), Zeitschrift für Orthopädie und Unfallchirurgie, 10.1055/a-0997-6922, 158, 05, (481-489), (2020).
  • Efficacy of photodynamic therapy for the treatment of Bowen’s disease: An updated systematic review and meta-analysis of randomized controlled trials, Photodiagnosis and Photodynamic Therapy, 10.1016/j.pdpdt.2020.102037, (102037), (2020).
  • Am I out of control? The application of statistical process control charts to children's surgery, Journal of Pediatric Surgery, 10.1016/j.jpedsurg.2019.12.027, (2020).
  • The Dutch Lung Cancer Audit: Nationwide quality of care evaluation of lung cancer patients, Lung Cancer, 10.1016/j.lungcan.2020.08.011, (2020).
  • Failure to Cure in Patients Undergoing Surgery for Esophageal Carcinoma: Hospital of Surgery Influences Prospects for Cure, Annals of Surgery, 10.1097/SLA.0000000000004178, 272, 5, (744-750), (2020).
  • Observational cohort study to determine the degree and causes of variation in the rate of surgery or primary endocrine therapy in older women with operable breast cancer, European Journal of Surgical Oncology, 10.1016/j.ejso.2020.09.029, (2020).
  • Evaluating the long-term effects of a data-driven approach to reduce variation in emergency department pathology investigations: study protocol for evaluation of the NSW Health Pathology Atlas of variation, BMJ Open, 10.1136/bmjopen-2020-039437, 10, 10, (e039437), (2020).
  • Center-Specific Risk-Adjusted Standardized Mortality Rates on Continuous Ambulatory Peritoneal Dialysis in China, Peritoneal Dialysis International: Journal of the International Society for Peritoneal Dialysis, 10.3747/pdi.2018.00085, 38, 2_suppl, (36-44), (2020).
  • The Pennsylvania Trauma Outcomes Study Risk-Adjusted Mortality Model: Results of a Statewide Benchmarking Program, The American Surgeon, 10.1177/000313481708300516, 83, 5, (445-452), (2020).
  • Routinely measuring symptom burden and health-related quality of life in dialysis patients: first results from the Dutch registry of patient-reported outcome measures, Clinical Kidney Journal, 10.1093/ckj/sfz192, (2020).
  • Therapist personality traits as predictors of psychological treatment outcomes, Psychotherapy Research, 10.1080/10503307.2020.1731927, (1-14), (2020).
  • Funnel plots of patient‐reported outcomes to evaluate health‐care quality: Basic principles, pitfalls and considerations, Nephrology, 10.1111/nep.13761, 0, 0, (2020).
  • CATTELL'S parallel proportional profiles, Journal of Chemometrics, 10.1002/cem.3235, 0, 0, (2020).
  • Frontline Science: CD40 signaling restricts RNA virus replication in Mϕs, leading to rapid innate immune control of acute virus infection, Journal of Leukocyte Biology, 10.1002/JLB.4HI0420-285RR, 0, 0, (2020).
  • Angiotensin‐Converting Enzyme Inhibitor Prescription for Patients With Single Ventricle Physiology Enrolled in the NPC‐QIC Registry, Journal of the American Heart Association, 10.1161/JAHA.119.014823, (2020).
  • Using monitors to monitor ecological restoration: Presence may not indicate persistence, Austral Ecology, 10.1111/aec.12905, 0, 0, (2020).
  • Using EQ-5D Data to Measure Hospital Performance: Are General Population Values Distorting Patients’ Choices?, Medical Decision Making, 10.1177/0272989X20927705, (0272989X2092770), (2020).
  • A small number of surgeons outside the control-limit: an observational study based on 9,482 cases and 208 surgeons performing primary total hip arthroplasties in western Sweden, Acta Orthopaedica, 10.1080/17453674.2020.1772584, (1-6), (2020).
  • The orchard plot: Cultivating a forest plot for use in ecology, evolution, and beyond, Research Synthesis Methods, 10.1002/jrsm.1424, 0, 0, (2020).
  • Impact on survival of modelling increased surgical resection rates in patients with non-small-cell lung cancer and cardiovascular comorbidities: a VICORI study, British Journal of Cancer, 10.1038/s41416-020-0869-8, (2020).
  • Outcome quality standards for surgery of colorectal liver metastasis, Langenbeck's Archives of Surgery, 10.1007/s00423-020-01908-w, (2020).
  • Delta Studies: Expanding the Concept of Deviance Studies to Design More Effective Improvement Interventions, Journal of General Internal Medicine, 10.1007/s11606-020-06199-x, (2020).
  • Association Between Olfactory Impairment and Disease Severity and Duration in Parkinson's Disease, Movement Disorders Clinical Practice, 10.1002/mdc3.13028, 0, 0, (2020).
  • Validation and quality measurements for STS, EuroSCORE II and a regional risk model in Brazilian patients, PLOS ONE, 10.1371/journal.pone.0238737, 15, 9, (e0238737), (2020).
  • Monitoring of high-yield and periodical processes in health care, Health Care Management Science, 10.1007/s10729-020-09514-4, (2020).
  • Variations in concerns reported on the patient concerns inventory in patients with head and neck cancer from different health settings across the world, Head & Neck, 10.1002/hed.26027, 42, 3, (498-512), (2019).
  • Understanding FEV1 for the purpose of cystic fibrosis registry comparisons: Does bias in annual review FEV1 affect between‐centre comparison within the UK? An analysis of registry data, Journal of Evaluation in Clinical Practice, 10.1111/jep.13097, 26, 1, (229-235), (2019).
  • Pericardiobiliary fistula causing acute pericarditis and tamponade following extensive cytoreductive surgery, ANZ Journal of Surgery, 10.1111/ans.15417, 90, 5, (919-920), (2019).
  • Intraoperative nerve monitoring in endocrine surgery: prevalence and reasons for its use in Australia and New Zealand, ANZ Journal of Surgery, 10.1111/ans.15536, 90, 5, (867-871), (2019).
  • Transhepatic gallbladder: an unusual variation, ANZ Journal of Surgery, 10.1111/ans.15339, 90, 5, (885-886), (2019).
  • Endometriosis: a rare cause of acute appendicitis in pregnancy, ANZ Journal of Surgery, 10.1111/ans.15454, 90, 5, (935-937), (2019).
  • Diagnostic accuracy of procalcitonin for the early diagnosis of anastomotic leakage after colorectal surgery: a meta‐analysis, ANZ Journal of Surgery, 10.1111/ans.15291, 90, 5, (675-680), (2019).
  • Case report of nasopharyngeal rhabdomyosarcoma causing obstructive sleep apnoea, ANZ Journal of Surgery, 10.1111/ans.15369, 90, 5, (904-905), (2019).
  • Significant inter‐ and intra‐laboratory variation in grading of invasive breast cancer: A nationwide study of 33,043 patients in the Netherlands, International Journal of Cancer, 10.1002/ijc.32330, 146, 3, (769-780), (2019).
  • Gastric Bypass Versus Sleeve Gastrectomy, Annals of Surgery, 10.1097/SLA.0000000000003279, 272, 2, (326-333), (2019).
  • Performance assessment as an application of causal inference, Journal of the Royal Statistical Society: Series A (Statistics in Society), 10.1111/rssa.12529, 183, 4, (1363-1385), (2019).
  • Rectal cancer in old age –is it appropriately managed? Evidence from population-based analysis of routine data across the English national health service, European Journal of Surgical Oncology, 10.1016/j.ejso.2019.01.005, (2019).
  • Factors affecting adenoma detection rate in a national flexible sigmoidoscopy screening programme: a retrospective analysis, The Lancet Gastroenterology & Hepatology, 10.1016/S2468-1253(18)30387-X, (2019).
  • Social outcomes for adults with a history of childhood-onset epilepsy: A systematic review and meta-analysis, Epilepsy & Behavior, 10.1016/j.yebeh.2019.01.012, 92, (297-305), (2019).
  • Evaluation of the effectiveness of hip and knee implant models used in Catalonia: a protocol for a prospective registry-based study, Journal of Orthopaedic Surgery and Research, 10.1186/s13018-019-1087-z, 14, 1, (2019).
  • 2018 AATS/ACC/SCAI/STS Expert Consensus Systems of Care Document: Operator and Institutional Recommendations and Requirements for Transcatheter Aortic Valve Replacement, The Annals of Thoracic Surgery, 10.1016/j.athoracsur.2018.07.001, 107, 2, (650-684), (2019).
  • Regional comparison of cancer incidence, mortality, and survival on the level of federal states in Germany using funnel plots, European Journal of Cancer Prevention, 10.1097/CEJ.0000000000000446, 28, 3, (234-242), (2019).
  • Challenges of Measuring and Assuring Quality for Transcatheter Aortic Valve Replacement, Journal of the American College of Cardiology, 10.1016/j.jacc.2018.09.092, 73, 3, (336-339), (2019).
  • 2018 AATS/ACC/SCAI/STS Expert Consensus Systems of Care Document: Operator and Institutional Recommendations and Requirements for Transcatheter Aortic Valve Replacement, Journal of the American College of Cardiology, 10.1016/j.jacc.2018.07.002, 73, 3, (340-374), (2019).
  • Spatial and temporal variations in Spain in the standardised ratio of in-hospital mortality due to colorectal cancer, 2008–2014, BMC Cancer, 10.1186/s12885-019-5502-y, 19, 1, (2019).
  • Tips for calculating and displaying risk-standardized hospital outcomes in Stata, The Stata Journal: Promoting communications on statistics and Stata, 10.1177/1536867X19854021, 19, 2, (477-496), (2019).
  • Assessing local chlamydia screening performance by combining survey and administrative data to account for differences in local population characteristics, Scientific Reports, 10.1038/s41598-019-43521-y, 9, 1, (2019).
  • VARIATION IN DOCUMENTED INHALATION INJURY RATES FOLLOWING BURN INJURY IN AUSTRALIA AND NEW ZEALAND, Injury, 10.1016/j.injury.2019.11.023, (2019).
  • Epidemiologic Studies of Medical Devices, Pharmacoepidemiology, 10.1002/9781119413431, (496-523), (2019).
  • Variation in the Use of Resection for Colorectal Cancer Liver Metastases, Annals of Surgery, 10.1097/SLA.0000000000003534, 270, 5, (892-898), (2019).
  • Home care nursing after elective vascular surgery: an opportunity to reduce emergency department visits and hospital readmission, BMJ Quality & Safety, 10.1136/bmjqs-2018-009161, 28, 11, (901-907), (2019).
  • Family satisfaction with critical care in the UK: a multicentre cohort study, BMJ Open, 10.1136/bmjopen-2019-028956, 9, 8, (e028956), (2019).
  • Predictors of hospitalizations for diabetes in Germany: an ecological study on a small-area scale, Public Health, 10.1016/j.puhe.2019.08.003, 177, (112-119), (2019).
  • Continuous Monitoring of Transplant Center Performance, Transplantation, 10.1097/TP.0000000000002591, 103, 9, (1935-1944), (2019).
  • 2018 AATS/ACC/SCAI/STS expert consensus systems of care document: Operator and institutional recommendations and requirements for transcatheter aortic valve replacement, Catheterization and Cardiovascular Interventions, 10.1002/ccd.27811, 93, 3, (E153-E184), (2019).
  • Insight from data analytics with an automotive aftermarket SME, Quality and Reliability Engineering International, 10.1002/qre.2529, 35, 5, (1396-1407), (2019).
  • Nationwide improvement in outcomes of emergency admission for ulcerative colitis in England, 2005‐2013, Alimentary Pharmacology & Therapeutics, 10.1111/apt.15315, 50, 2, (176-192), (2019).
  • Variation in outcomes and use of laparoscopy in elective inguinal hernia repair, BJS Open, 10.1002/bjs5.50158, 3, 4, (466-475), (2019).
  • Validation of the acute cholecystectomy rate as a quality indicator for emergency general surgery using the SWORD database, The Annals of The Royal College of Surgeons of England, 10.1308/rcsann.2019.0042, 101, 6, (422-427), (2019).
  • Risk-Adjusting Key Outcome Measures in a Clinical Quality PCI Registry, JACC: Cardiovascular Interventions, 10.1016/j.jcin.2019.07.002, 12, 19, (1966-1975), (2019).
  • See more

The full text of this article hosted at iucr.org is unavailable due to technical difficulties.