Predicting Complications of Major Head and Neck Oncological Surgery: an Evaluation of the ACS NSQIP Surgical Risk Calculator

Overview

Journal J Otolaryngol Head Neck Surg

Publisher Sage Publications

Specialties General Surgery
Otorhinolaryngology

Date 2018 Mar 24

PMID 29566750

Citations 19

Authors

Peter S Vosler

Mario Orsini

Danny J Enepekides

Kevin M Higgins

Affiliations

Soon will be listed here.

Abstract

Background: The American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) universal surgical risk calculator is an online tool intended to improve the informed consent process and surgical decision-making. The risk calculator uses a database of information from 585 hospitals to predict a patient's risk of developing specific postoperative outcomes.

Methods: Patient records at a major Canadian tertiary care referral center between July 2015 and March 2017 were reviewed for surgical cases including one of six major head and neck oncologic surgeries: total thyroidectomy, total laryngectomy, hemiglossectomy, partial glossectomy, laryngopharyngectomy, and composite resection. Preoperative information for 107 patients was entered into the risk calculator and compared to observed postoperative outcomes. Statistical analysis of the risk calculator was completed for the entire study population, for stratification by procedure, and by utilization of microvascular reconstruction. Accuracy was assessed using the ratio of predicted to observed outcomes, Receiver Operating Characteristics (ROC), Brier score, and the Wilcoxon signed-ranked test.

Results: The risk calculator accurately predicted the incidences for 11 of 12 outcomes for patients that did not undergo free flap reconstruction (NFF group), but was less accurate for patients that underwent free flap reconstruction (FF group). Length of stay (LOS) analysis showed similar results, with predicted and observed LOS statistically different in the overall population and FF group analyses (p = 0.001 for both), but not for the NFF group analysis (p = 0.764). All outcomes in the NFF group, when analyzed for calibration, met the threshold value (Brier scores < 0.09). Risk predictions for 8 of 12, and 10 of 12 outcomes were adequately calibrated in the FF group and the overall study population, respectively. Analyses by procedure were excellent, with the risk calculator showing adequate calibration for 7 of 8 procedural categories and adequate discrimination for all calculable categories (6 of 6).

Conclusion: The NSQIP-RC demonstrated efficacy for predicting postoperative complications in head and neck oncology surgeries that do not require microvascular reconstruction. The predictive value of the metric can be improved by inclusion of several factors important for risk stratification in head and neck oncology.

Citing Articles

A meta-analysis of the American college of surgeons risk calculator's predictive accuracy among different surgical sub-specialties.

Goodwin A, Kurapaty S, Inglis J, Divi S, Patel A, Hsu W Surg Pract Sci. 2025; 16():100238.

PMID: 39845345 PMC: 11749946. DOI: 10.1016/j.sipas.2024.100238.

Major Perioperative Cardiac Risk Assessment: A Review for Cardio-Oncologists and Perioperative Physicians.

Johnson E, Monsour R, Hafez O, Kotha R, Ackerman R Clin Pract. 2024; 14(3):906-914.

PMID: 38804403 PMC: 11130950. DOI: 10.3390/clinpract14030071.

How surgeons use risk calculators and non-clinical factors for informed consent and shared decision making: A qualitative study.

Panton J, Beaulieu-Jones B, Marwaha J, Woods A, Nakikj D, Gehlenborg N Am J Surg. 2023; 226(5):660-667.

PMID: 37468387 PMC: 10592325. DOI: 10.1016/j.amjsurg.2023.07.017.

Effective Communication of Personalized Risks and Patient Preferences During Surgical Informed Consent Using Data Visualization: Qualitative Semistructured Interview Study With Patients After Surgery.

Gisladottir U, Nakikj D, Jhunjhunwala R, Panton J, Brat G, Gehlenborg N JMIR Hum Factors. 2022; 9(2):e29118.

PMID: 35486432 PMC: 9107059. DOI: 10.2196/29118.

Inaccurate Risk Assessment by the ACS NSQIP Risk Calculator in Aortic Surgery.

Hers T, van Schaik J, Keekstra N, Putter H, Hamming J, van der Vorst J J Clin Med. 2021; 10(22).

PMID: 34830708 PMC: 8618691. DOI: 10.3390/jcm10225426.

References

Veeravagu A, Li A, Swinney C, Tian L, Moraff A, Azad T . Predicting complication risk in spine surgery: a prospective analysis of a novel risk assessment tool. J Neurosurg Spine. 2017; 27(1):81-91. DOI: 10.3171/2016.12.SPINE16969. View

le Nobel G, Higgins K, Enepekides D . Predictors of complications of free flap reconstruction in head and neck surgery: Analysis of 304 free flap reconstruction procedures. Laryngoscope. 2012; 122(5):1014-9. DOI: 10.1002/lary.22454. View

Velanovich V . Testing the Ability of the NSQIP Risk Calculator to Predict Laryngectomy Complications. JAMA Otolaryngol Head Neck Surg. 2016; 142(10):979-980. DOI: 10.1001/jamaoto.2016.1861. View

Samson P, Robinson C, Bradley J, Lee A, Broderick S, Kreisel D . The National Surgical Quality Improvement Program risk calculator does not adequately stratify risk for patients with clinical stage I non-small cell lung cancer. J Thorac Cardiovasc Surg. 2015; 151(3):697-705.e1. PMC: 4957558. DOI: 10.1016/j.jtcvs.2015.08.058. View

Bilimoria K, Liu Y, Paruch J, Zhou L, Kmiecik T, Ko C . Development and evaluation of the universal ACS NSQIP surgical risk calculator: a decision aid and informed consent tool for patients and surgeons. J Am Coll Surg. 2013; 217(5):833-42.e1-3. PMC: 3805776. DOI: 10.1016/j.jamcollsurg.2013.07.385. View

Vaziri S, Wilson J, Abbatematteo J, Kubilis P, Chakraborty S, Kshitij K . Predictive performance of the American College of Surgeons universal risk calculator in neurosurgical patients. J Neurosurg. 2017; 128(3):942-947. DOI: 10.3171/2016.11.JNS161377. View

Nam R, Kattan M, Chin J, Trachtenberg J, Singal R, Rendon R . Prospective multi-institutional study evaluating the performance of prostate cancer risk calculators. J Clin Oncol. 2011; 29(22):2959-64. DOI: 10.1200/JCO.2010.32.6371. View

Peterson B, Ghahramani M, Harris S, Suchniak-Mussari K, Bedi G, Bulathsinghala C . Usefulness of the Myocardial Infarction and Cardiac Arrest Calculator as a Discriminator of Adverse Cardiac Events After Elective Hip and Knee Surgery. Am J Cardiol. 2016; 117(12):1992-5. DOI: 10.1016/j.amjcard.2016.03.050. View

Gupta P, Franck C, Miller W, Gupta H, Forse R . Development and validation of a bariatric surgery morbidity risk calculator using the prospective, multicenter NSQIP dataset. J Am Coll Surg. 2011; 212(3):301-9. DOI: 10.1016/j.jamcollsurg.2010.11.003. View

10.

Adegboyega T, Borgert A, Lambert P, Jarman B . Applying the National Surgical Quality Improvement Program risk calculator to patients undergoing colorectal surgery: theory vs reality. Am J Surg. 2016; 213(1):30-35. DOI: 10.1016/j.amjsurg.2016.04.011. View

11.

Prasad K, Nelson B, Deig C, Schneider A, Moore M . ACS NSQIP Risk Calculator: An Accurate Predictor of Complications in Major Head and Neck Surgery?. Otolaryngol Head Neck Surg. 2016; 155(5):740-742. DOI: 10.1177/0194599816655976. View

12.

Cohen M, Liu Y, Ko C, Hall B . An Examination of American College of Surgeons NSQIP Surgical Risk Calculator Accuracy. J Am Coll Surg. 2017; 224(5):787-795.e1. DOI: 10.1016/j.jamcollsurg.2016.12.057. View

13.

Massoumi R, Trevino C, Webb T . Postoperative Complications of Laparoscopic Cholecystectomy for Acute Cholecystitis: A Comparison to the ACS-NSQIP Risk Calculator and the Tokyo Guidelines. World J Surg. 2016; 41(4):935-939. DOI: 10.1007/s00268-016-3816-3. View

14.

Hajian-Tilaki K . Receiver Operating Characteristic (ROC) Curve Analysis for Medical Diagnostic Test Evaluation. Caspian J Intern Med. 2013; 4(2):627-35. PMC: 3755824. View

15.

Slump J, Ferguson P, Wunder J, Griffin A, Hoekstra H, Bagher S . Can the ACS-NSQIP surgical risk calculator predict post-operative complications in patients undergoing flap reconstruction following soft tissue sarcoma resection?. J Surg Oncol. 2016; 114(5):570-575. DOI: 10.1002/jso.24357. View

16.

Steyerberg E, Vickers A, Cook N, Gerds T, Gonen M, Obuchowski N . Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2009; 21(1):128-38. PMC: 3575184. DOI: 10.1097/EDE.0b013e3181c30fb2. View

17.

Liu Y, Cohen M, Hall B, Ko C, Bilimoria K . Evaluation and Enhancement of Calibration in the American College of Surgeons NSQIP Surgical Risk Calculator. J Am Coll Surg. 2016; 223(2):231-9. DOI: 10.1016/j.jamcollsurg.2016.03.040. View

18.

Teoh D, Halloway R, Heim J, Vogel R, Rivard C . Evaluation of the American College of Surgeons National Surgical Quality Improvement Program Surgical Risk Calculator in Gynecologic Oncology Patients Undergoing Minimally Invasive Surgery. J Minim Invasive Gynecol. 2016; 24(1):48-54. PMC: 6614862. DOI: 10.1016/j.jmig.2016.10.005. View

19.

Cohen M, Bilimoria K, Ko C, Hall B . Development of an American College of Surgeons National Surgery Quality Improvement Program: morbidity and mortality risk calculator for colorectal surgery. J Am Coll Surg. 2009; 208(6):1009-16. DOI: 10.1016/j.jamcollsurg.2009.01.043. View

20.

Winoker J, Paulucci D, Anastos H, Waingankar N, Abaza R, Eun D . Predicting Complications Following Robot-Assisted Partial Nephrectomy with the ACS NSQIP Universal Surgical Risk Calculator. J Urol. 2017; 198(4):803-809. DOI: 10.1016/j.juro.2017.04.021. View