Likely Change Indexes Improve Estimates of Individual Change on Patient-reported Outcomes

Overview

Journal Qual Life Res

Specialties Pharmacology
Rehabilitation Medicine

Date 2022 Aug 3

PMID 35921034

Authors

John Devin Peipert

Ron D Hays

David Cella

Affiliations

Soon will be listed here.

Abstract

Purpose: Individual change on a patient-reported outcome (PRO) measure can be assessed by statistical significance and meaningfulness to patients. We explored the relationship between these two criteria by varying the confidence levels of the coefficient of repeatability (CR) on the Patient-Reported Outcomes Measurement Information System (R) Physical Function (PF) 10a (PF10a) measure.

Methods: In a sample of 1129 adult cancer patients, we estimated individual-change thresholds on the PF10a from baseline to 6 weeks later with the CR at 50%, 68%, and 95% confidence. We also assessed agreement with group- and individual-level thresholds from anchor-based methods [mean change and receiver operating characteristic (ROC) curve] using a PF-specific patient global impression of change (PGIC).

Results: CRs at 50%, 68%, and 95% confidence were 3, 4, and 7 raw score points, respectively. The ROC- and mean-change-based thresholds for deterioration were -4 and -6; for improvement they were both 2. Kappas for agreement between anchor-based thresholds and CRs for deterioration ranged between κ = 0.65 and 1.00, while for improvement, they ranged between 0.35 and 0.83. Agreement between the PGIC and all CRs always fell below "good" (κ < 0.40) for deterioration (0.30-0.33) and were lower for improvement (0.16-0.28).

Conclusions: In comparison to the CR at 95% confidence, CRs at 50% and 68% confidence (considered likely change indexes) have the advantage of maximizing the proportion of patients appropriately classified as changed according to statistical significance and meaningfulness.

Citing Articles

Commentary on "When (Not) to Rely on the Reliable Change Index".

Hays R, Reise S Clin Psychol (New York). 2025; 31(3):367-369.

PMID: 40028657 PMC: 11870641. DOI: 10.1037/cps0000210.

Clinically meaningful improvements after gene therapy for aromatic L-amino acid decarboxylase deficiency (AADCd) in the Peabody Developmental Motor Scale, Second Edition (PDMS-2) and correlation with Bayley-III scores and motor milestones.

Hwu W, Lee H, Peipert J, Zhang R, Werner C, Sierra J Orphanet J Rare Dis. 2025; 20(1):58.

PMID: 39920752 PMC: 11806574. DOI: 10.1186/s13023-025-03584-9.

Significant individual change should be used as a lower bound for anchor based estimates of meaningful change on patient-reported outcome scores.

Peipert J, Cella D, Hays R Qual Life Res. 2024; 33(12):3223-3228.

PMID: 39340723 PMC: 11599412. DOI: 10.1007/s11136-024-03788-9.

Validity, reliability, responsiveness, and clinically meaningful change threshold estimates of the National Comprehensive Cancer Network-Functional Assessment of Cancer Therapy-Breast Cancer Symptom Index (NFBSI-16).

Clarke N, Wong B, Lawrance R, Ingelgard A, Griebsch I, Cella D J Patient Rep Outcomes. 2024; 8(1):97.

PMID: 39145900 PMC: 11327234. DOI: 10.1186/s41687-024-00776-y.

Linking QuickDASH and PROMIS Upper-Extremity Computer-Adaptive Test Scores in Hand Surgery: A Crosswalk Study.

Cizik A, Zhang C, Presson A, Randall D, Kazmers N J Hand Surg Am. 2024; 49(7):664-674.

PMID: 38795102 PMC: 11451358. DOI: 10.1016/j.jhsa.2024.04.004.

References

Terwee C, Peipert J, Chapman R, Lai J, Terluin B, Cella D . Minimal important change (MIC): a conceptual clarification and systematic review of MIC estimates of PROMIS measures. Qual Life Res. 2021; 30(10):2729-2754. PMC: 8481206. DOI: 10.1007/s11136-021-02925-y. View

Terluin B, Eekhout I, Terwee C . The anchor-based minimal important change, based on receiver operating characteristic analysis or predictive modeling, may need to be adjusted for the proportion of improved patients. J Clin Epidemiol. 2017; 83:90-100. DOI: 10.1016/j.jclinepi.2016.12.015. View

Terluin B, Eekhout I, Terwee C, de Vet H . Minimal important change (MIC) based on a predictive modeling approach was more precise than MIC based on ROC analysis. J Clin Epidemiol. 2015; 68(12):1388-96. DOI: 10.1016/j.jclinepi.2015.03.015. View

Norman G, Stratford P, Regehr G . Methodological problems in the retrospective computation of responsiveness to change: the lesson of Cronbach. J Clin Epidemiol. 1997; 50(8):869-79. DOI: 10.1016/s0895-4356(97)00097-8. View

Hays R, Brodsky M, Johnston M, Spritzer K, Hui K . Evaluating the statistical significance of health-related quality-of-life change in individual patients. Eval Health Prof. 2005; 28(2):160-71. DOI: 10.1177/0163278705275339. View

Moinpour C, Donaldson G, Davis K, Potosky A, Jensen R, Gralow J . The challenge of measuring intra-individual change in fatigue during cancer treatment. Qual Life Res. 2016; 26(2):259-271. PMC: 5274585. DOI: 10.1007/s11136-016-1372-9. View

King M, Dueck A, Revicki D . Can Methods Developed for Interpreting Group-level Patient-reported Outcome Data be Applied to Individual Patient Management?. Med Care. 2019; 57 Suppl 5 Suppl 1:S38-S45. PMC: 6467500. DOI: 10.1097/MLR.0000000000001111. View

Jacobson N, Truax P . Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. J Consult Clin Psychol. 1991; 59(1):12-9. DOI: 10.1037//0022-006x.59.1.12. View

Cella D, Bullinger M, Scott C, Barofsky I . Group vs individual approaches to understanding the clinical significance of differences or changes in quality of life. Mayo Clin Proc. 2002; 77(4):384-92. DOI: 10.4065/77.4.384. View

10.

Donaldson G . Patient-reported outcomes and the mandate of measurement. Qual Life Res. 2008; 17(10):1303-13. DOI: 10.1007/s11136-008-9408-4. View

11.

Lee M, Schalet B, Cella D, Yost K, Dueck A, Novotny P . Establishing a common metric for patient-reported outcomes in cancer patients: linking patient reported outcomes measurement information system (PROMIS), numerical rating scale, and patient-reported outcomes version of the common terminology.... J Patient Rep Outcomes. 2020; 4(1):106. PMC: 7728866. DOI: 10.1186/s41687-020-00271-0. View

12.

Jensen R, Potosky A, Reeve B, Hahn E, Cella D, Fries J . Validation of the PROMIS physical function measures in a diverse US population-based cohort of cancer patients. Qual Life Res. 2015; 24(10):2333-44. PMC: 5079641. DOI: 10.1007/s11136-015-0992-9. View

13.

Oken M, Creech R, Tormey D, Horton J, Davis T, McFadden E . Toxicity and response criteria of the Eastern Cooperative Oncology Group. Am J Clin Oncol. 1982; 5(6):649-55. View

14.

Hays R, Peipert J . Between-group minimally important change versus individual treatment responders. Qual Life Res. 2021; 30(10):2765-2772. PMC: 8204732. DOI: 10.1007/s11136-021-02897-z. View

15.

Youden W . Index for rating diagnostic tests. Cancer. 1950; 3(1):32-5. DOI: 10.1002/1097-0142(1950)3:1<32::aid-cncr2820030106>3.0.co;2-3. View

16.

Terwee C, Terluin B, Knol D, de Vet H . Combining clinical relevance and statistical significance for evaluating quality of life changes in the individual patient. J Clin Epidemiol. 2011; 64(12):1465-7. DOI: 10.1016/j.jclinepi.2011.06.015. View

17.

Terwee C, Roorda L, Knol D, de Boer M, de Vet H . Linking measurement error to minimal important change of patient-reported outcomes. J Clin Epidemiol. 2009; 62(10):1062-7. DOI: 10.1016/j.jclinepi.2008.10.011. View

18.

Coon C, Cook K . Moving from significance to real-world meaning: methods for interpreting change in clinical outcome assessment scores. Qual Life Res. 2017; 27(1):33-40. DOI: 10.1007/s11136-017-1616-3. View

19.

Segawa E, Schalet B, Cella D . A comparison of computer adaptive tests (CATs) and short forms in terms of accuracy and number of items administrated using PROMIS profile. Qual Life Res. 2019; 29(1):213-221. DOI: 10.1007/s11136-019-02312-8. View

20.

Cella D, Riley W, Stone A, Rothrock N, Reeve B, Yount S . The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005-2008. J Clin Epidemiol. 2010; 63(11):1179-94. PMC: 2965562. DOI: 10.1016/j.jclinepi.2010.04.011. View