External Validation of the National Surgical Quality Improvement Program Calculator Utilizing a Single Institutional Experience for Adult Spinal Deformity Corrective Surgery

Overview

Journal Int J Spine Surg

Date 2023 Feb 15

PMID 36792364

Authors

Sara Naessig

Katherine Pierce

Waleed Ahmad

Lara Passfall

Oscar Krol

Nicholas A Kummer

Tyler Williamson

Bailey Imbo

Peter Tretiakov

Kevin Moattari

Rachel Joujon-Roche

Jack Zhong

Eaman Balouch

Brooke OConnell

Constance Maglaras

Bassel Diebo

Renaud Lafage

Virginie Lafage

Shaleen Vira

Steven Hale

Michael Gerling

Themistocles Protopsaltis

Aaron Buckland

Peter G Passias

Affiliations

Soon will be listed here.

Abstract

Background: Identify the external applicability of the American College of Surgeons' National Surgical Quality Improvement Program (NSQIP) risk calculator in the setting of adult spinal deformity (ASD) and subsets of patients based on deformity and frailty status.

Methods: ASD patients were isolated in our single-center database and analyzed for the shared predictive variables displayed in the NSQIP calculator. Patients were stratified by frailty (not frail <0.03, frail 0.3-0.5, severely frail >0.5), deformity [T1 pelvic angle (TPA) > 30, pelvic incidence minus lumbar lordosis (PI-LL) > 20], and reoperation status. Brier scores were calculated for each variable to validate the calculator's predictability in a single center's database (Quality). External validity of the calculator in our ASD patients was assessed via Hosmer-Lemeshow test, which identified whether the differences between observed and expected proportions are significant.

Results: A total of 1606 ASD patients were isolated from the Quality database (48.7 years, 63.8% women, 25.8 kg/m); 33.4% received decompressions, and 100% received a fusion. For each subset of ASD patients, the calculator predicted lower outcome rates than what was identified in the Quality database. The calculator showed poor predictability for frail, deformed, and reoperation patients for the category "any complication" because they had Brier scores closer to 1. External validity of the calculator in each stratified patient group identified that the calculator was not valid, displaying values >0.05.

Conclusion: The NSQIP calculator was not a valid calculator in our single institutional database. It is unable to comment on surgical complications such as return to operating room, surgical site infection, urinary tract infection, and cardiac complications that are typically associated with poor patient outcomes. Physicians should not base their surgical plan solely on the NSQIP calculator but should consider multiple preoperative risk assessment tools.

Level Of Evidence: 3:

Citing Articles

Outcomes and Complications After Elective Thoracic and Lumbar Spinal Fusion in Elderly Patients: A Comparison of Methods to Predict Adverse Events.

Ramos O, Mueller B, Mehbod A, Carlson B Global Spine J. 2024; :21925682241300977.

PMID: 39563006 PMC: 11577552. DOI: 10.1177/21925682241300977.

References

Veeravagu A, Li A, Swinney C, Tian L, Moraff A, Azad T . Predicting complication risk in spine surgery: a prospective analysis of a novel risk assessment tool. J Neurosurg Spine. 2017; 27(1):81-91. DOI: 10.3171/2016.12.SPINE16969. View

Hamilton D, Lane J, Gaston P, Patton J, MacDonald D, Simpson A . What determines patient satisfaction with surgery? A prospective cohort study of 4709 patients following total joint replacement. BMJ Open. 2013; 3(4). PMC: 3641464. DOI: 10.1136/bmjopen-2012-002525. View

Bilimoria K, Liu Y, Paruch J, Zhou L, Kmiecik T, Ko C . Development and evaluation of the universal ACS NSQIP surgical risk calculator: a decision aid and informed consent tool for patients and surgeons. J Am Coll Surg. 2013; 217(5):833-42.e1-3. PMC: 3805776. DOI: 10.1016/j.jamcollsurg.2013.07.385. View

Cho S, Bridwell K, Lenke L, Yi J, Pahys J, P Zebala L . Major complications in revision adult deformity surgery: risk factors and clinical outcomes with 2- to 7-year follow-up. Spine (Phila Pa 1976). 2011; 37(6):489-500. DOI: 10.1097/BRS.0b013e3182217ab5. View

Nam R, Kattan M, Chin J, Trachtenberg J, Singal R, Rendon R . Prospective multi-institutional study evaluating the performance of prostate cancer risk calculators. J Clin Oncol. 2011; 29(22):2959-64. DOI: 10.1200/JCO.2010.32.6371. View

Schwab F, Farcy J, Bridwell K, Berven S, Glassman S, Harrast J . A clinical impact classification of scoliosis in the adult. Spine (Phila Pa 1976). 2006; 31(18):2109-14. DOI: 10.1097/01.brs.0000231725.38943.ab. View

Mogal H, Fino N, Clark C, Shen P . Comparison of observed to predicted outcomes using the ACS NSQIP risk calculator in patients undergoing pancreaticoduodenectomy. J Surg Oncol. 2016; 114(2):157-62. PMC: 5591630. DOI: 10.1002/jso.24276. View

Miller E, Neuman B, Jain A, Daniels A, Ailon T, Sciubba D . An assessment of frailty as a tool for risk stratification in adult spinal deformity surgery. Neurosurg Focus. 2017; 43(6):E3. DOI: 10.3171/2017.10.FOCUS17472. View

Khanduja K, Scales D, Adhikari N . Pay for performance in the intensive care unit--opportunity or threat?. Crit Care Med. 2009; 37(3):852-8. DOI: 10.1097/CCM.0b013e3181962b0b. View

10.

Wang X, Hu Y, Zhao B, Su Y . Predictive validity of the ACS-NSQIP surgical risk calculator in geriatric patients undergoing lumbar surgery. Medicine (Baltimore). 2017; 96(43):e8416. PMC: 5671873. DOI: 10.1097/MD.0000000000008416. View

11.

Gupta P, Gupta H, Sundaram A, Kaushik M, Fang X, Miller W . Development and validation of a risk calculator for prediction of cardiac risk after surgery. Circulation. 2011; 124(4):381-7. DOI: 10.1161/CIRCULATIONAHA.110.015701. View

12.

Gray D, Deyo R, Kreuter W, Mirza S, Heagerty P, Comstock B . Population-based trends in volumes and rates of ambulatory lumbar spine surgery. Spine (Phila Pa 1976). 2006; 31(17):1957-63. DOI: 10.1097/01.brs.0000229148.63418.c1. View

13.

Lafage V, Schwab F, Vira S, Patel A, Ungar B, Farcy J . Spino-pelvic parameters after surgery can be predicted: a preliminary formula and validation of standing alignment. Spine (Phila Pa 1976). 2011; 36(13):1037-45. DOI: 10.1097/BRS.0b013e3181eb9469. View

14.

Osorio J, Scheer J, Ames C . Predictive modeling of complications. Curr Rev Musculoskelet Med. 2016; 9(3):333-7. PMC: 4958386. DOI: 10.1007/s12178-016-9354-7. View

15.

Cologne K, Hwang G, Senagore A . Cost of practice in a tertiary/quaternary referral center: is it sustainable?. Tech Coloproctol. 2014; 18(11):1035-9. DOI: 10.1007/s10151-014-1175-3. View

16.

Ali R, Schwalb J, Nerenz D, Antoine H, Rubinfeld I . Use of the modified frailty index to predict 30-day morbidity and mortality from spine surgery. J Neurosurg Spine. 2016; 25(4):537-541. DOI: 10.3171/2015.10.SPINE14582. View

17.

Searle S, Mitnitski A, Gahbauer E, Gill T, Rockwood K . A standard procedure for creating a frailty index. BMC Geriatr. 2008; 8:24. PMC: 2573877. DOI: 10.1186/1471-2318-8-24. View

18.

Kwok A, Lipsitz S, Bader A, Gawande A . Are targeted preoperative risk prediction tools more powerful? A test of models for emergency colon surgery in the very elderly. J Am Coll Surg. 2011; 213(2):220-5. DOI: 10.1016/j.jamcollsurg.2011.04.025. View

19.

Finks J, Kole K, Yenumula P, English W, Krause K, Carlin A . Predicting risk for serious complications with bariatric surgery: results from the Michigan Bariatric Surgery Collaborative. Ann Surg. 2011; 254(4):633-40. DOI: 10.1097/SLA.0b013e318230058c. View

20.

McCarthy M, Singh P, Maslak J, Nayak R, Jenkins T, Hsu W . Can the American College of Surgeons Risk Calculator Predict 30-Day Complications After Cervical Spine Surgery?. Clin Spine Surg. 2019; 32(9):357-362. DOI: 10.1097/BSD.0000000000000890. View