Cumulative Funnel Plots for the Early Detection of Interoperator Variation: Retrospective Database Analysis of Observed Versus Predicted Results of Percutaneous Coronary Intervention

Overview

Journal BMJ

Specialty General Medicine

Date 2008 Mar 28

PMID 18367500

Citations 6

Authors

Babu Kunadian

Joel Dunning

Anthony P Roberts

Robert Morley

Darragh Twomey

James A Hall

Andrew G C Sutton

Robert A Wright

Douglas F Muir

Mark A de Belder

Affiliations

Soon will be listed here.

Abstract

Objective: To use funnel plots and cumulative funnel plots to compare in-hospital outcome data for operators undertaking percutaneous coronary interventions with predicted results derived from a validated risk score to allow for early detection of variation in performance.

Design: Analysis of prospectively collected data.

Setting: Tertiary centre NHS hospital in the north east of England.

Participants: Five cardiologists carrying out percutaneous coronary interventions between January 2003 and December 2006.

Main Outcome Measures: In-hospital major adverse cardiovascular and cerebrovascular events (in-hospital death, Q wave myocardial infarction, emergency coronary artery bypass graft surgery, and cerebrovascular accident) analysed against the logistic north west quality improvement programme predicted risk, for each operator. Results are displayed as funnel plots summarising overall performance for each operator and cumulative funnel plots for an individual operator's performance on a case series basis.

Results: The funnel plots for 5198 patients undergoing percutaneous coronary interventions showed an average observed rate for major adverse cardiovascular and cerebrovascular events of 1.96% overall. This was below the predicted risk of 2.06% by the logistic north west quality improvement programme risk score. Rates of in-hospital major adverse cardiovascular and cerebrovascular events for all operators were within the 3sigma upper control limit of 2.75% and 2sigma upper warning limit of 2.49%.

Conclusion: The overall in-hospital major adverse cardiovascular and cerebrovascular events rates were under the predicted event rate. In-hospital rates after percutaneous coronary intervention procedure can be monitored successfully using funnel and cumulative funnel plots with 3sigma control limits to display and publish each operator's outcomes. The upper warning limit (2sigma control limit) could be used for internal monitoring. The main advantage of these charts is their transparency, as they show observed and predicted events separately. By this approach individual operators can monitor their own performance, using the predicted risk for their patients but in a way that is compatible with benchmarking to colleagues, encapsulated by the funnel plot. This methodology is applicable regardless of variations in individual operator case volume and case mix.

Citing Articles

Benchmarking survival outcomes: A funnel plot for survival data.

Putter H, Eikema D, de Wreede L, McGrath E, Sanchez-Ortega I, Saccardi R Stat Methods Med Res. 2022; 31(6):1171-1183.

PMID: 35257603 PMC: 9245152. DOI: 10.1177/09622802221084130.

Impact of guideline-recommended dietitian assessments on weight gain in infants with cystic fibrosis.

Ong T, Onchiri F, Britto M, Heltshe S, Kessler L, Seid M J Cyst Fibros. 2021; 21(1):115-122.

PMID: 34454845 PMC: 8844148. DOI: 10.1016/j.jcf.2021.08.005.

Application of Variable Life Adjusted Displays (VLAD) on Victorian Admitted Episodes Dataset (VAED).

Andrianopoulos N, Jolley D, Evans S, Brand C, Cameron P BMC Health Serv Res. 2012; 12:278.

PMID: 22925089 PMC: 3576249. DOI: 10.1186/1472-6963-12-278.

Severe head injury in children: intensive care unit activity and mortality in England and Wales.

Tasker R, Fleming T, Young A, Morris K, Parslow R Br J Neurosurg. 2010; 25(1):68-77.

PMID: 21083365 PMC: 3038595. DOI: 10.3109/02688697.2010.538770.

Doomed to repeat the successes of the past: history is best forgotten for repeated choices with nonstationary payoffs.

Rakow T, Miler K Mem Cognit. 2009; 37(7):985-1000.

PMID: 19744938 DOI: 10.3758/MC.37.7.985.

References

WILLIAMS D, Holubkov R, Yeh W, Bourassa M, Block P, Coady P . Percutaneous coronary intervention in the current era compared with 1985-1986: the National Heart, Lung, and Blood Institute Registries. Circulation. 2000; 102(24):2945-51. DOI: 10.1161/01.cir.102.24.2945. View

Bridgewater B . Mortality data in adult cardiac surgery for named surgeons: retrospective examination of prospectively collected data on coronary artery surgery and aortic valve replacement. BMJ. 2005; 330(7490):506-10. PMC: 552809. DOI: 10.1136/bmj.330.7490.506. View

Kunadian B, Dunning J, Das R, Roberts A, Morley R, Turley A . External validation of established risk adjustment models for procedural complications after percutaneous coronary intervention. Heart. 2007; 94(8):1012-8. DOI: 10.1136/hrt.2007.129197. View

Detre K, Yeh W, Kelsey S, Williams D, Desvigne-Nickens P, Holmes Jr D . Has improvement in PTCA intervention affected long-term prognosis? The NHLBI PTCA Registry experience. Circulation. 1995; 91(12):2868-75. DOI: 10.1161/01.cir.91.12.2868. View

Bridgewater B, Grayson A, Jackson M, Brooks N, Grotte G, Keenan D . Surgeon specific mortality in adult cardiac surgery: comparison between crude and risk stratified data. BMJ. 2003; 327(7405):13-7. PMC: 164230. DOI: 10.1136/bmj.327.7405.13. View

Kimmel S, Berlin J, Strom B, Laskey W . Development and validation of simplified predictive index for major complications in contemporary percutaneous transluminal coronary angioplasty practice. The Registry Committee of the Society for Cardiac Angiography and Interventions. J Am Coll Cardiol. 1995; 26(4):931-8. DOI: 10.1016/0735-1097(95)00294-4. View

Spiegelhalter D . Funnel plots for institutional comparison. Qual Saf Health Care. 2002; 11(4):390-1. PMC: 1757996. DOI: 10.1136/qhc.11.4.390-a. View

Block P, Peterson E, Krone R, Kesler K, Hannan E, OConnor G . Identification of variables needed to risk adjust outcomes of coronary interventions: evidence-based guidelines for efficient data collection. J Am Coll Cardiol. 1998; 32(1):275-82. DOI: 10.1016/s0735-1097(98)00208-3. View

King 3rd S, Aversano T, Ballard W, Beekman 3rd R, Cowley M, Ellis S . ACCF/AHA/SCAI 2007 update of the Clinical Competence Statement on Cardiac Interventional Procedures: a report of the American College of Cardiology Foundation/American Heart Association/American College of Physicians Task Force on Clinical.... Circulation. 2007; 116(1):98-124. DOI: 10.1161/CIRCULATIONAHA.107.185159. View

10.

Spiegelhalter D . Funnel plots for comparing institutional performance. Stat Med. 2004; 24(8):1185-202. DOI: 10.1002/sim.1970. View

11.

Gale C, Roberts A, Batin P, Hall A . Funnel plots, performance variation and the Myocardial Infarction National Audit Project 2003-2004. BMC Cardiovasc Disord. 2006; 6:34. PMC: 1555633. DOI: 10.1186/1471-2261-6-34. View

12.

Narins C, Dozier A, Ling F, Zareba W . The influence of public reporting of outcome data on medical decision making by physicians. Arch Intern Med. 2005; 165(1):83-7. DOI: 10.1001/archinte.165.1.83. View

13.

OConnor G, Malenka D, Quinton H, Robb J, Kellett Jr M, Shubrooks S . Multivariate prediction of in-hospital mortality after percutaneous coronary interventions in 1994-1996. Northern New England Cardiovascular Disease Study Group. J Am Coll Cardiol. 1999; 34(3):681-91. DOI: 10.1016/s0735-1097(99)00267-3. View

14.

Colson M, Bolsin S . The use of statistical process control methods in monitoring clinical performance. Int J Qual Health Care. 2003; 15(5):445. DOI: 10.1093/intqhc/mzg070. View

15.

Spiegelhalter D, Grigg O, Kinsman R, Treasure T . Risk-adjusted sequential probability ratio tests: applications to Bristol, Shipman and adult cardiac surgery. Int J Qual Health Care. 2003; 15(1):7-13. DOI: 10.1093/intqhc/15.1.7. View

16.

Wolfe R, Bolsin S, Colson M, Stow P . Monitoring the rate of re-exploration for excessive bleeding after cardiac surgery in adults. Qual Saf Health Care. 2007; 16(3):192-6. PMC: 2464986. DOI: 10.1136/qshc.2004.012435. View

17.

Grayson A, Moore R, Jackson M, Rathore S, Sastry S, Gray T . Multivariate prediction of major adverse cardiac events after 9914 percutaneous coronary interventions in the north west of England. Heart. 2005; 92(5):658-63. PMC: 1860907. DOI: 10.1136/hrt.2005.066415. View

18.

Bridgewater B, Grayson A, Brooks N, Grotte G, Fabri B, Au J . Has the publication of cardiac surgery outcome data been associated with changes in practice in northwest England: an analysis of 25,730 patients undergoing CABG surgery under 30 surgeons over eight years. Heart. 2007; 93(6):744-8. PMC: 1955202. DOI: 10.1136/hrt.2006.106393. View

19.

Teasdale G . Learning from Bristol: report of the public inquiry into children's heart surgery at Bristol Royal Infirmary 1984-1995. Br J Neurosurg. 2002; 16(3):211-6. DOI: 10.1080/02688690220148815. View

20.

Moscucci M, OConnor G, ELLIS S, Malenka D, Sievers J, Bates E . Validation of risk adjustment models for in-hospital percutaneous transluminal coronary angioplasty mortality on an independent data set. J Am Coll Cardiol. 1999; 34(3):692-7. DOI: 10.1016/s0735-1097(99)00266-1. View