Construct Validation of Machine Learning in the Prediction of Short-term Postoperative Complications Following Total Shoulder Arthroplasty

Background: We aimed to demonstrate that supervised machine learning (ML) models can better predict postoperative complications after total shoulder arthroplasty (TSA) than comorbidity indices.

Methods: The American College of Surgeons-National Surgical Quality Improvement Program database was queried from 2005-2017 for TSA cases. Training and validation sets were created by randomly assigning 80% and 20% of the data set. Included variables were age, body mass index (BMI), operative time, smoking status, comorbidities, diagnosis, and preoperative hematocrit and albumin. Complications included any adverse event, transfusion, extended length of stay (>3 days), surgical site infection, return to the operating room, deep vein thrombosis or pulmonary embolism, and readmission. Each SML algorithm was compared with one another and to a baseline model using American Society of Anesthesiologists (ASA) classification. Model strength was evaluated by calculating the area under the receiver operating characteristic curve (AUC) and the positive predictive value (PPV) of complications.

Results: We identified a total of 17,119 TSA cases. Mean age, BMI, and length of stay were 69.5 ± 9.6 years, 31.1 ± 6.8, and 2.0 ± 2.2 days. Percentage hematocrit, BMI, and operative time were of highest importance in outcome prediction. SML algorithms outperformed ASA classification models for predicting any adverse event (71.0% vs. 63.0%), transfusion (77.0% vs. 64.0%), extended length of stay (68.0% vs. 60.0%), surgical site infection (65.0% vs. 58.0%), return to the operating room (59.0% vs. 54.0%), and readmission (64.0% vs. 58.0%). SML algorithms demonstrated the greatest PPV for any adverse event (62.5%), extended length of stay (61.4%), transfusion (52.2%), and readmission (10.1%). ASA classification had a 0.0% PPV for complications.

Conclusion: With continued validation, intelligent models could calculate patient-specific risk for complications to adjust perioperative care and site of surgery.