Recalibration and External Validation of the Risk Analysis Index: A Surgical Frailty Assessment Tool

Overview

Journal Ann Surg

Specialty General Surgery

Date 2019 Mar 26

PMID 30907757

Citations 67

Authors

Shipra Arya

Patrick Varley

Ada Youk

Jeffrey D Borrebach

Sebastian Perez

Nader N Massarweh

Jason M Johanning

Daniel E Hall

Affiliations

Soon will be listed here.

Abstract

Objective And Background: The Risk Analysis Index (RAI) predicts 30-, 180-, and 365-day mortality based on variables constitutive of frailty. Initially validated, in a single-center Veteran hospital, we sought to improve model performance by recalibrating the RAI in a large, veteran surgical registry, and to externally validate it in both a national surgical registry and a cohort of surgical patients for whom RAI was measured prospectively before surgery.

Methods: The RAI was recalibrated among development and confirmation samples within the Veterans Affairs Surgical Quality Improvement Program (VASQIP; 2010-2014; N = 480,731) including major, elective noncardiac surgery patients to create the revised RAI (RAI-rev), comparing discrimination and calibration. The model was tested externally in the American College of Surgeons National Surgical Quality Improvement Program dataset (NSQIP; 2005-2014; N = 1,391,785), and in a prospectively collected cohort from the Nebraska Western Iowa Health Care System VA (NWIHCS; N = 6,856).

Results: Recalibrating the RAI significantly improved discrimination for 30-day [c = 0.84-0.86], 180-day [c = 0.81-0.84], and 365-day mortality [c = 0.78-0.82] (P < 0.001 for all) in VASQIP. The RAI-rev also had markedly better calibration (median absolute difference between observed and predicted 180-day mortality: decreased from 8.45% to 1.23%). RAI-rev was highly predictive of 30-day mortality (c = 0.87) in external validation with excellent calibration (median absolute difference between observed and predicted 30-day mortality: 0.6%). The discrimination was highly robust in men (c = 0.85) and women (c = 0.89). Discrimination also improved in the prospectively measured cohort from NWIHCS for 180-day mortality [c = 0.77 to 0.80] (P < 0.001).

Conclusions: The RAI-rev has improved discrimination and calibration as a frailty-screening tool in surgical patients. It has robust external validity in men and women across a wide range of surgical settings and available for immediate implementation for risk assessment and counseling in preoperative patients.

Citing Articles

Orthopedic frailty risk stratification (OFRS): a systematic review of the frailty indices predicting adverse outcomes in orthopedics.

Gupta N, Dunivin F, Chmait H, Smitterberg C, Buttar A, Fazal-Ur-Rehman M J Orthop Surg Res. 2025; 20(1):247.

PMID: 40051013 PMC: 11887260. DOI: 10.1186/s13018-025-05609-2.

The Risk Analysis Index Has Superior Discrimination Compared With the Modified Frailty Index-5 in Predicting Worse Postoperative Outcomes for the Octogenarian Neurosurgical Patient.

Yocky A, Owodunni O, Courville E, Kazim S, Schmidt M, Gearhart S Neurosurg Pract. 2025; 4(3):e00044.

PMID: 39958794 PMC: 11809970. DOI: 10.1227/neuprac.0000000000000044.

Agreement Between Provider-Completed and Patient-Completed Preoperative Frailty Screening Using the Clinical Risk Analysis Index: Cross-Sectional Questionnaire Study.

Khalighi M, Thomas A, Brown K, Ritchey K JMIR Perioper Med. 2025; 8:e66440.

PMID: 39928399 PMC: 11851030. DOI: 10.2196/66440.

Diagnosis of frailty and implications on surgical process in the elderly: A narrative review.

Aceto P, Schipa C, Luca E, Cambise C, Galletta C, Tommasino C Eur J Anaesthesiol Intensive Care. 2025; 2(6):e0041.

PMID: 39916728 PMC: 11798398. DOI: 10.1097/EA9.0000000000000041.

The Surgical Pause: The Importance of Measuring Frailty and Taking Action to Address Identified Frailty.

Hall D, Hagan D, Ashcraft L, Wilson M, Arya S, Johanning J Jt Comm J Qual Patient Saf. 2025; 51(3):167-177.

PMID: 39799070 PMC: 11867859. DOI: 10.1016/j.jcjq.2024.11.011.

References

Hall D, Arya S, Schmid K, Blaser C, Carlson M, Bailey T . Development and Initial Validation of the Risk Analysis Index for Measuring Frailty in Surgical Populations. JAMA Surg. 2016; 152(2):175-182. PMC: 7140150. DOI: 10.1001/jamasurg.2016.4202. View

Robinson T, Eiseman B, Wallace J, Church S, McFann K, Pfister S . Redefining geriatric preoperative assessment using frailty, disability and co-morbidity. Ann Surg. 2009; 250(3):449-55. DOI: 10.1097/SLA.0b013e3181b45598. View

Kwok A, Semel M, Lipsitz S, Bader A, Barnato A, Gawande A . The intensity and variation of surgical care at the end of life: a retrospective cohort study. Lancet. 2011; 378(9800):1408-13. DOI: 10.1016/S0140-6736(11)61268-3. View

Aguayo G, Donneau A, Vaillant M, Schritz A, Franco O, Stranges S . Agreement Between 35 Published Frailty Scores in the General Population. Am J Epidemiol. 2017; 186(4):420-434. PMC: 5860330. DOI: 10.1093/aje/kwx061. View

Schwarze M, Brasel K, Mosenthal A . Beyond 30-day mortality: aligning surgical quality with outcomes that patients value. JAMA Surg. 2014; 149(7):631-2. PMC: 4295201. DOI: 10.1001/jamasurg.2013.5143. View

Porock D, Parker-Oliver D, Petroski G, Rantz M . The MDS Mortality Risk Index: The evolution of a method for predicting 6-month mortality in nursing home residents. BMC Res Notes. 2010; 3:200. PMC: 2913927. DOI: 10.1186/1756-0500-3-200. View

Gani F, Canner J, Pawlik T . Use of the Modified Frailty Index in the American College of Surgeons National Surgical Improvement Program Database: Highlighting the Problem of Missing Data. JAMA Surg. 2016; 152(2):205-207. DOI: 10.1001/jamasurg.2016.3479. View

Khuri S, Henderson W, Daley J, Jonasson O, Jones R, Campbell Jr D . Successful implementation of the Department of Veterans Affairs' National Surgical Quality Improvement Program in the private sector: the Patient Safety in Surgery study. Ann Surg. 2008; 248(2):329-36. DOI: 10.1097/SLA.0b013e3181823485. View

Rockwood K, Andrew M, Mitnitski A . A comparison of two approaches to measuring frailty in elderly people. J Gerontol A Biol Sci Med Sci. 2007; 62(7):738-43. DOI: 10.1093/gerona/62.7.738. View

10.

Robinson T, Wu D, Stiegmann G, Moss M . Frailty predicts increased hospital and six-month healthcare cost following colorectal surgery in older adults. Am J Surg. 2011; 202(5):511-4. PMC: 3346286. DOI: 10.1016/j.amjsurg.2011.06.017. View

11.

Subramaniam S, Aalberg J, Soriano R, Divino C . New 5-Factor Modified Frailty Index Using American College of Surgeons NSQIP Data. J Am Coll Surg. 2017; 226(2):173-181.e8. DOI: 10.1016/j.jamcollsurg.2017.11.005. View

12.

Kopecky K, Urbach D, Schwarze M . Risk Calculators and Decision Aids Are Not Enough for Shared Decision Making. JAMA Surg. 2018; 154(1):3-4. DOI: 10.1001/jamasurg.2018.2446. View

13.

Anaya D, Johanning J, Spector S, Katlic M, Perrino A, Feinleib J . Summary of the panel session at the 38th Annual Surgical Symposium of the Association of VA Surgeons: what is the big deal about frailty?. JAMA Surg. 2014; 149(11):1191-7. DOI: 10.1001/jamasurg.2014.2064. View

14.

Khuri S, Daley J, Henderson W, Hur K, Demakis J, AUST J . The Department of Veterans Affairs' NSQIP: the first national, validated, outcome-based, risk-adjusted, and peer-controlled program for the measurement and enhancement of the quality of surgical care. National VA Surgical Quality Improvement Program. Ann Surg. 1998; 228(4):491-507. PMC: 1191523. DOI: 10.1097/00000658-199810000-00006. View

15.

Wick E, Finlayson E . Frailty-Going From Measurement to Action. JAMA Surg. 2017; 152(8):757-758. DOI: 10.1001/jamasurg.2017.1050. View

16.

Rockwood K, Mitnitski A . Frailty defined by deficit accumulation and geriatric medicine defined by frailty. Clin Geriatr Med. 2010; 27(1):17-26. DOI: 10.1016/j.cger.2010.08.008. View

17.

Sternberg S, Schwartz A, Karunananthan S, Bergman H, Clarfield A . The identification of frailty: a systematic literature review. J Am Geriatr Soc. 2011; 59(11):2129-38. DOI: 10.1111/j.1532-5415.2011.03597.x. View

18.

Berian J, Zhou L, Hornor M, Russell M, Cohen M, Finlayson E . Optimizing Surgical Quality Datasets to Care for Older Adults: Lessons from the American College of Surgeons NSQIP Geriatric Surgery Pilot. J Am Coll Surg. 2017; 225(6):702-712.e1. DOI: 10.1016/j.jamcollsurg.2017.08.012. View

19.

Velanovich V, Antoine H, Swartz A, Peters D, Rubinfeld I . Accumulating deficits model of frailty and postoperative mortality and morbidity: its application to a national database. J Surg Res. 2013; 183(1):104-10. DOI: 10.1016/j.jss.2013.01.021. View

20.

Suskind A, Finlayson E . A Call for Frailty Screening in the Preoperative Setting. JAMA Surg. 2016; 152(3):240-241. PMC: 5352541. DOI: 10.1001/jamasurg.2016.4256. View