Use of Computerized Adaptive Testing to Develop More Concise Patient-Reported Outcome Measures

Overview

Journal JB JS Open Access

Date 2020 Apr 21

PMID 32309761

Citations 12

Authors

Liam T Kane

Surena Namdari

Otho R Plummer

Pedro Beredjiklian

Alexander Vaccaro

Joseph A Abboud

Affiliations

Soon will be listed here.

Abstract

Background: Patient-reported outcome measures (PROMs) are essential tools that are used to assess health status and treatment outcomes in orthopaedic care. Use of PROMs can burden patients with lengthy and cumbersome questionnaires. Predictive models using machine learning known as offer a potential solution. The purpose of this study was to evaluate the ability of CAT to improve efficiency of the Veterans RAND 12 Item Health Survey (VR-12) by decreasing the question burden while maintaining the accuracy of the outcome score.

Methods: A previously developed CAT model was applied to the responses of 19,523 patients who had completed a full VR-12 survey while presenting to 1 of 5 subspecialty orthopaedic clinics. This resulted in the calculation of both a full-survey and CAT-model physical component summary score (PCS) and mental component summary score (MCS). Several analyses compared the accuracy of the CAT model scores with that of the full scores by comparing the means and standard deviations, calculating a Pearson correlation coefficient and intraclass correlation coefficient, plotting the frequency distributions of the 2 score sets and the score differences, and performing a Bland-Altman assessment of scoring patterns.

Results: The CAT model required 4 fewer questions to be answered by each subject (33% decrease in question burden). The mean PCS was 1.3 points lower in the CAT model than with the full VR-12 (41.5 ± 11.0 versus 42.8 ± 10.4), and the mean MCS was 0.3 point higher (57.3 ± 9.4 versus 57.0 ± 9.6). The Pearson correlation coefficients were 0.97 for PCS and 0.98 for MCS, and the intraclass correlation coefficients were 0.96 and 0.97, respectively. The frequency distribution of the CAT and full scores showed significant overlap for both the PCS and the MCS. The difference between the CAT and full scores was less than the minimum clinically important difference (MCID) in >95% of cases for the PCS and MCS.

Conclusions: The application of CAT to the VR-12 survey demonstrated an ability to lessen the response burden for patients with a negligible effect on score integrity.

Citing Articles

Optimizing Oxford Shoulder Scores with computerized adaptive testing reduces redundancy while maintaining precision.

Barakat A, Evans J, Gibbons C, Singh H Bone Joint Res. 2024; 13(8):392-400.

PMID: 39097982 PMC: 11298256. DOI: 10.1302/2046-3758.138.BJR-2023-0412.R1.

Seamless Integration of Computer-Adaptive Patient Reported Outcomes into an Electronic Health Record.

Nolla K, Rasmussen L, Rothrock N, Butt Z, Bass M, Davis K Appl Clin Inform. 2023; 15(1):145-154.

PMID: 38154472 PMC: 10881259. DOI: 10.1055/a-2235-9557.

Validation of the cognitive section of the Penn computerized adaptive test for neurocognitive and clinical psychopathology assessment (CAT-CCNB).

Di Sandro A, Moore T, Zoupou E, Kennedy K, Lopez K, Ruparel K Brain Cogn. 2023; 174:106117.

PMID: 38128447 PMC: 10799332. DOI: 10.1016/j.bandc.2023.106117.

Research Trends of Patient-Reported Outcome Measures in Orthopedic Medical Practices: A Bibliometric and Visualized Study.

Jin H, He M, Xie W, Xiong Z, Deng Z, Li Y Medicina (Kaunas). 2023; 59(9).

PMID: 37763783 PMC: 10536719. DOI: 10.3390/medicina59091664.

Danish Linguistic Validation and Cultural Adaptation of the LIMB-Q Kids.

Jonsson C, Poulsen L, Rolfing J, Chhina H, Cooper A, Sorensen J Children (Basel). 2023; 10(7).

PMID: 37508603 PMC: 10377942. DOI: 10.3390/children10071107.

References

Bland J, Altman D . Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986; 1(8476):307-10. View

Parker S, Godil S, Shau D, Mendenhall S, McGirt M . Assessment of the minimum clinically important difference in pain, disability, and quality of life after anterior cervical discectomy and fusion: clinical article. J Neurosurg Spine. 2012; 18(2):154-60. DOI: 10.3171/2012.10.SPINE12312. View

McHorney C, Ware Jr J, Lu J, Sherbourne C . The MOS 36-item Short-Form Health Survey (SF-36): III. Tests of data quality, scaling assumptions, and reliability across diverse patient groups. Med Care. 1994; 32(1):40-66. DOI: 10.1097/00005650-199401000-00004. View

Bienstock D, Snyder D, Kroshus T, Ahn A, Koenig K, Molloy I . Relationship Between Baseline Patient-reported Outcomes and Demographic, Psychosocial, and Clinical Characteristics: A Retrospective Study. J Am Acad Orthop Surg Glob Res Rev. 2019; 3(5):e039. PMC: 6553630. DOI: 10.5435/JAAOSGlobal-D-19-00039. View

Gornet M, Copay A, Sorensen K, Schranck F . Assessment of health-related quality of life in spine treatment: conversion from SF-36 to VR-12. Spine J. 2018; 18(7):1292-1297. DOI: 10.1016/j.spinee.2018.02.023. View

Selim A, Rogers W, Fleishman J, Qian S, Fincke B, Rothendler J . Updated U.S. population standard for the Veterans RAND 12-item Health Survey (VR-12). Qual Life Res. 2008; 18(1):43-52. DOI: 10.1007/s11136-008-9418-2. View

Schalet B, Rothrock N, Hays R, Kazis L, Cook K, Rutsohn J . Linking Physical and Mental Health Summary Scores from the Veterans RAND 12-Item Health Survey (VR-12) to the PROMIS(®) Global Health Scale. J Gen Intern Med. 2015; 30(10):1524-30. PMC: 4579239. DOI: 10.1007/s11606-015-3453-9. View

Bessette M, Westermann R, Davis A, Farrow L, Hagen M, Miniaci A . Predictors of Pain and Function Before Knee Arthroscopy. Orthop J Sports Med. 2019; 7(5):2325967119844265. PMC: 6537074. DOI: 10.1177/2325967119844265. View

Plummer O, Abboud J, Bell J, Murthi A, Romeo A, Singh P . A concise shoulder outcome measure: application of computerized adaptive testing to the American Shoulder and Elbow Surgeons Shoulder Assessment. J Shoulder Elbow Surg. 2019; 28(7):1273-1280. DOI: 10.1016/j.jse.2018.11.068. View

10.

Kazis L, Selim A, Rogers W, Qian S, Brazier J . Monitoring outcomes for the Medicare Advantage program: methods and application of the VR-12 for evaluation of plans. J Ambul Care Manage. 2012; 35(4):263-276. DOI: 10.1097/JAC.0b013e318267468f. View

11.

Bents E, Brady P, Adams C, Tokish J, Higgins L, Denard P . Patient-Reported Outcomes of Knotted and Knotless Glenohumeral Labral Repairs Are Equivalent. Am J Orthop (Belle Mead NJ). 2018; 46(6):279-283. View

12.

Chien T, Wu H, Wang W, Castillo R, Chou W . Reduction in patient burdens with graphical computerized adaptive testing on the ADL scale: tool development and simulation. Health Qual Life Outcomes. 2009; 7:39. PMC: 2688502. DOI: 10.1186/1477-7525-7-39. View

13.

Thigpen C, Floyd S, Chapman C, Tokish J, Kissenberth M, Hawkins R . Comparison of Surgeon Performance of Rotator Cuff Repair: Risk Adjustment Toward a More Accurate Performance Measure. J Bone Joint Surg Am. 2018; 100(24):2110-2117. DOI: 10.2106/JBJS.18.00211. View

14.

Wolfe F, Michaud K . Proposed metrics for the determination of rheumatoid arthritis outcome and treatment success and failure. J Rheumatol. 2008; 36(1):27-33. DOI: 10.3899/jrheum.080591. View

15.

Selim A, Rogers W, Qian S, Rothendler J, Kent E, Kazis L . A new algorithm to build bridges between two patient-reported health outcome instruments: the MOS SF-36® and the VR-12 Health Survey. Qual Life Res. 2018; 27(8):2195-2206. DOI: 10.1007/s11136-018-1850-3. View

16.

Brogan A, DeMuro C, Barrett A, DAlessio D, Bal V, Hogue S . Payer Perspectives on Patient-Reported Outcomes in Health Care Decision Making: Oncology Examples. J Manag Care Spec Pharm. 2017; 23(2):125-134. PMC: 10397630. DOI: 10.18553/jmcp.2017.23.2.125. View

17.

Zhou L, Natarajan M, Miller B, Gagnier J . Establishing Minimal Important Differences for the VR-12 and SANE Scores in Patients Following Treatment of Rotator Cuff Tears. Orthop J Sports Med. 2018; 6(7):2325967118782159. PMC: 6077909. DOI: 10.1177/2325967118782159. View

18.

McIntyre L, Bishai S, Brown 3rd P, Bushnell B, Trenhaile S . Patient-Reported Outcomes After Use of a Bioabsorbable Collagen Implant to Treat Partial and Full-Thickness Rotator Cuff Tears. Arthroscopy. 2019; 35(8):2262-2271. DOI: 10.1016/j.arthro.2019.02.019. View

19.

Kazis L, Selim A, Rogers W, Ren X, Lee A, Miller D . Dissemination of methods and results from the veterans health study: final comments and implications for future monitoring strategies within and outside the veterans healthcare system. J Ambul Care Manage. 2006; 29(4):310-9. DOI: 10.1097/00004479-200610000-00007. View

20.

Lapin B, Kinzy T, Thompson N, Krishnaney A, Katzan I . Accuracy of Linking VR-12 and PROMIS Global Health Scores in Clinical Practice. Value Health. 2018; 21(10):1226-1233. DOI: 10.1016/j.jval.2018.03.011. View