Application of a Bayesian Graded Response Model to Characterize Areas of Disagreement Between Clinician and Patient Grading of Symptomatic Adverse Events

Overview

Journal J Patient Rep Outcomes

Specialty Health Services

Date 2018 Dec 6

PMID 30515599

Citations 2

Authors

Thomas M Atkinson

Bryce B Reeve

Amylou C Dueck

Antonia V Bennett

Tito R Mendoza

Lauren J Rogak

Ethan Basch

Yuelin Li

Affiliations

Soon will be listed here.

Abstract

Background: Traditional concordance metrics have shortcomings based on dataset characteristics (e.g., multiple attributes rated, missing data); therefore it is necessary to explore supplemental approaches to quantifying agreement between independent assessments. The purpose of this methodological paper is to apply an Item Response Theory (IRT) -based framework to an existing dataset that included unidimensional clinician and multiple attribute patient ratings of symptomatic adverse events (AEs), and explore the utility of this method in patient-reported outcome (PRO) and health-related quality of life (HRQOL) research.

Methods: Data were derived from a National Cancer Institute-sponsored study examining the validity of a measurement system (PRO-CTCAE) for patient self-reporting of AEs in cancer patients receiving treatment (N = 940). AEs included 13 multiple attribute patient-reported symptoms that had corresponding unidimensional clinician AE grades. A Bayesian IRT Model was fitted to calculate the latent grading thresholds between raters. The posterior mean values of the model-fitted item responses were calculated to represent model-based AE grades obtained from patients and clinicians.

Results: Model-based AE grades showed a general pattern of clinician underestimation relative to patient-graded AEs. However, the magnitude of clinician underestimation was associated with AE severity, such that clinicians' underestimation was more pronounced for moderate/very severe model-estimated AEs, and less so with mild AEs.

Conclusions: The Bayesian IRT approach reconciles multiple symptom attributes and elaborates on the patterns of clinician-patient non-concordance beyond that provided by traditional metrics. This IRT-based technique may be used as a supplemental tool to detect and characterize nuanced differences in patient-, clinician-, and proxy-based ratings of HRQOL and patient-centered outcomes.

Trial Registration: ClinicalTrials.gov NCT01031641 . Registered 1 December 2009.

Citing Articles

A Comparison of Patient- and Clinician-Reported Acute Toxic Effects During Radiation Therapy for Primary Breast Cancer.

Lapen K, King C, Braunstein L, Khan A, Kamrava M, Gillespie E Int J Radiat Oncol Biol Phys. 2022; 114(2):301-309.

PMID: 35675851 PMC: 10281649. DOI: 10.1016/j.ijrobp.2022.05.041.

Congruence of patient- and clinician-reported toxicity in women receiving chemotherapy for early breast cancer.

Nyrop K, Deal A, Reeve B, Basch E, Chen Y, Park J Cancer. 2020; 126(13):3084-3093.

PMID: 32315091 PMC: 7931261. DOI: 10.1002/cncr.32898.

References

Uebersax J . Modeling approaches for the analysis of observer agreement. Invest Radiol. 1992; 27(9):738-43. DOI: 10.1097/00004424-199209000-00017. View

Preen D, Holman C, Lawrence D, Baynham N, Semmens J . Hospital chart review provided more accurate comorbidity information than data from a general practitioner survey or an administrative database. J Clin Epidemiol. 2004; 57(12):1295-304. DOI: 10.1016/j.jclinepi.2004.03.016. View

Basch E, Iasonos A, McDonough T, Barz A, Culkin A, Kris M . Patient versus clinician symptom reporting using the National Cancer Institute Common Terminology Criteria for Adverse Events: results of a questionnaire-based study. Lancet Oncol. 2006; 7(11):903-9. DOI: 10.1016/S1470-2045(06)70910-X. View

Bjorner J, Chang C, Thissen D, Reeve B . Developing tailored instruments: item banking and computerized adaptive assessment. Qual Life Res. 2007; 16 Suppl 1:95-108. DOI: 10.1007/s11136-007-9168-6. View

Shrout P, Fleiss J . Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979; 86(2):420-8. DOI: 10.1037//0033-2909.86.2.420. View

Baldwin P, Bernstein J, Wainer H . Hip psychometrics. Stat Med. 2009; 28(17):2277-92. DOI: 10.1002/sim.3616. View

Basch E, Jia X, Heller G, Barz A, Sit L, Fruscione M . Adverse symptom event reporting by patients vs clinicians: relationships with clinical outcomes. J Natl Cancer Inst. 2009; 101(23):1624-32. PMC: 2786917. DOI: 10.1093/jnci/djp386. View

Basch E . The missing voice of patients in drug-safety reporting. N Engl J Med. 2010; 362(10):865-9. PMC: 3031980. DOI: 10.1056/NEJMp0911494. View

Greimel E, Bjelic-Radisic V, Pfisterer J, Hilpert F, Daghofer F, Pujade-Lauraine E . Toxicity and quality of life outcomes in ovarian cancer patients participating in randomized controlled trials. Support Care Cancer. 2010; 19(9):1421-7. DOI: 10.1007/s00520-010-0969-8. View

10.

Neben-Wittich M, Atherton P, Schwartz D, Sloan J, Griffin P, Deming R . Comparison of provider-assessed and patient-reported outcome measures of acute skin toxicity during a Phase III trial of mometasone cream versus placebo during breast radiotherapy: the North Central Cancer Treatment Group (N06C4). Int J Radiat Oncol Biol Phys. 2010; 81(2):397-402. PMC: 3365545. DOI: 10.1016/j.ijrobp.2010.05.065. View

11.

Atkinson T, Li Y, Coffey C, Sit L, Shaw M, Lavene D . Reliability of adverse symptom event reporting by clinicians. Qual Life Res. 2011; 21(7):1159-64. PMC: 3633532. DOI: 10.1007/s11136-011-0031-4. View

12.

Patrick D, Burke L, Gwaltney C, Leidy N, Martin M, Molsen E . Content validity--establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO Good Research Practices Task Force report: part 2--assessing respondent understanding. Value Health. 2011; 14(8):978-88. DOI: 10.1016/j.jval.2011.06.013. View

13.

Bennett B, Park S, Lin C, Friedlander M, Kiernan M, Goldstein D . Impact of oxaliplatin-induced neuropathy: a patient perspective. Support Care Cancer. 2012; 20(11):2959-67. DOI: 10.1007/s00520-012-1428-5. View

14.

Xiao C, Polomano R, Bruner D . Comparison between patient-reported and clinician-observed symptoms in oncology. Cancer Nurs. 2012; 36(6):E1-E16. DOI: 10.1097/NCC.0b013e318269040f. View

15.

Hay J, Atkinson T, Reeve B, Mitchell S, Mendoza T, Willis G . Cognitive interviewing of the US National Cancer Institute's Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE). Qual Life Res. 2013; 23(1):257-69. PMC: 3896507. DOI: 10.1007/s11136-013-0470-1. View

16.

Basch E, Reeve B, Mitchell S, Clauser S, Minasian L, Dueck A . Development of the National Cancer Institute's patient-reported outcomes version of the common terminology criteria for adverse events (PRO-CTCAE). J Natl Cancer Inst. 2014; 106(9). PMC: 4200059. DOI: 10.1093/jnci/dju244. View

17.

Dueck A, Mendoza T, Mitchell S, Reeve B, Castro K, Rogak L . Validity and Reliability of the US National Cancer Institute's Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE). JAMA Oncol. 2015; 1(8):1051-9. PMC: 4857599. DOI: 10.1001/jamaoncol.2015.2639. View

18.

Bennett A, Dueck A, Mitchell S, Mendoza T, Reeve B, Atkinson T . Mode equivalence and acceptability of tablet computer-, interactive voice response system-, and paper-based administration of the U.S. National Cancer Institute's Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse.... Health Qual Life Outcomes. 2016; 14:24. PMC: 4759776. DOI: 10.1186/s12955-016-0426-6. View

19.

Atkinson T, Ryan S, Bennett A, Stover A, Saracino R, Rogak L . The association between clinician-based common terminology criteria for adverse events (CTCAE) and patient-reported outcomes (PRO): a systematic review. Support Care Cancer. 2016; 24(8):3669-76. PMC: 4919215. DOI: 10.1007/s00520-016-3297-9. View

20.

Atkinson T, Rogak L, Heon N, Ryan S, Shaw M, Stark L . Exploring differences in adverse symptom event grading thresholds between clinicians and patients in the clinical trial setting. J Cancer Res Clin Oncol. 2017; 143(4):735-743. PMC: 5354957. DOI: 10.1007/s00432-016-2335-9. View