COSMIN Methodology for Evaluating the Content Validity of Patient-reported Outcome Measures: a Delphi Study

Overview

Journal Qual Life Res

Specialties Pharmacology
Rehabilitation Medicine

Date 2018 Mar 19

PMID 29550964

Citations 618

Authors

C B Terwee

C A C Prinsen

A Chiarotto

M J Westerman

D L Patrick

J Alonso

L M Bouter

H C W de Vet

L B Mokkink

Affiliations

Soon will be listed here.

Abstract

Background: Content validity is the most important measurement property of a patient-reported outcome measure (PROM) and the most challenging to assess. Our aims were to: (1) develop standards for evaluating the quality of PROM development; (2) update the original COSMIN standards for assessing the quality of content validity studies of PROMs; (3) develop criteria for what constitutes good content validity of PROMs, and (4) develop a rating system for summarizing the evidence on a PROM's content validity and grading the quality of the evidence in systematic reviews of PROMs.

Methods: An online 4-round Delphi study was performed among 159 experts from 21 countries. Panelists rated the degree to which they (dis)agreed to proposed standards, criteria, and rating issues on 5-point rating scales ('strongly disagree' to 'strongly agree'), and provided arguments for their ratings.

Results: Discussion focused on sample size requirements, recording and field notes, transcribing cognitive interviews, and data coding. After four rounds, the required 67% consensus was reached on all standards, criteria, and rating issues. After pilot-testing, the steering committee made some final changes. Ten criteria for good content validity were defined regarding item relevance, appropriateness of response options and recall period, comprehensiveness, and comprehensibility of the PROM.

Discussion: The consensus-based COSMIN methodology for content validity is more detailed, standardized, and transparent than earlier published guidelines, including the previous COSMIN standards. This methodology can contribute to the selection and use of high-quality PROMs in research and clinical practice.

Citing Articles

New evidence for content validity of the German version of the Acute Cystitis Symptom Score: cognitive interview study among patients and experts.

Nestler S, Apfelbacher C, Ozkan E, Naber K, Piontek K World J Urol. 2025; 43(1):86.

PMID: 40085224 DOI: 10.1007/s00345-024-05406-2.

Young children (6-7 years) can meaningfully participate in cognitive interviews assessing comprehensibility in health-related quality of life domains: a qualitative study.

Gale V, Powell P, Carlton J Qual Life Res. 2025; .

PMID: 40044965 DOI: 10.1007/s11136-025-03940-z.

Critical evaluation of COSMIN scores in scales for mild cognitive impairment and Alzheimer's disease: A comprehensive review.

Peng X, Li R, Liang T, Xu S, Cao Y J Alzheimers Dis Rep. 2025; 8(1):1596-1610.

PMID: 40034351 PMC: 11863726. DOI: 10.1177/25424823241299023.

Development and preliminary validation of the Danish headache questionnaire.

Dissing K, Jensen R, Christensen H, Jensen M, Lauridsen H Chiropr Man Therap. 2025; 33(1):10.

PMID: 40011936 PMC: 11866720. DOI: 10.1186/s12998-025-00573-4.

Childbirth experience assessment tools based on COSMIN guidelines: A systematic review.

Cheng Y, Bai R, Shan S, Zhao X, Xia C Int J Nurs Sci. 2025; 12(1):89-95.

PMID: 39990987 PMC: 11846598. DOI: 10.1016/j.ijnss.2024.12.001.

References

Aaronson N, Alonso J, Burnam A, Lohr K, Patrick D, Perrin E . Assessing health status and quality-of-life instruments: attributes and review criteria. Qual Life Res. 2002; 11(3):193-205. DOI: 10.1023/a:1015291021312. View

Messick S . PERSONALITY MEASUREMENT AND THE ETHICS OF ASSESSMENT. Am Psychol. 1965; 20:136-42. DOI: 10.1037/h0021712. View

Mokkink L, Terwee C, Patrick D, Alonso J, Stratford P, Knol D . The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Qual Life Res. 2010; 19(4):539-49. PMC: 2852520. DOI: 10.1007/s11136-010-9606-8. View

Mokkink L, Terwee C, Patrick D, Alonso J, Stratford P, Knol D . The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. J Clin Epidemiol. 2010; 63(7):737-45. DOI: 10.1016/j.jclinepi.2010.02.006. View

Leidy N, Vernon M . Perspectives on patient-reported outcomes : content validity and qualitative research in a changing clinical trial environment. Pharmacoeconomics. 2008; 26(5):363-70. DOI: 10.2165/00019053-200826050-00002. View

Terwee C, Mokkink L, Knol D, Ostelo R, Bouter L, de Vet H . Rating the methodological quality in systematic reviews of studies on measurement properties: a scoring system for the COSMIN checklist. Qual Life Res. 2011; 21(4):651-7. PMC: 3323819. DOI: 10.1007/s11136-011-9960-1. View

Magasi S, Ryan G, Revicki D, Lenderking W, Hays R, Brod M . Content validity of patient-reported outcome measures: perspectives from a PROMIS meeting. Qual Life Res. 2011; 21(5):739-46. DOI: 10.1007/s11136-011-9990-8. View

Terwee C, Bot S, de Boer M, van der Windt D, Knol D, Dekker J . Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. 2006; 60(1):34-42. DOI: 10.1016/j.jclinepi.2006.03.012. View

Patrick D, Burke L, Gwaltney C, Leidy N, Martin M, Molsen E . Content validity--establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO Good Research Practices Task Force report: part 2--assessing respondent understanding. Value Health. 2011; 14(8):978-88. DOI: 10.1016/j.jval.2011.06.013. View

10.

Kuper A, Reeves S, Levinson W . An introduction to reading and appraising qualitative research. BMJ. 2008; 337:a288. DOI: 10.1136/bmj.a288. View

11.

Rothman M, Burke L, Erickson P, Leidy N, Patrick D, Petrie C . Use of existing patient-reported outcome (PRO) instruments and their modification: the ISPOR Good Research Practices for Evaluating and Documenting Content Validity for the Use of Existing Instruments and Their Modification PRO Task Force Report. Value Health. 2009; 12(8):1075-83. DOI: 10.1111/j.1524-4733.2009.00603.x. View

12.

Guyatt G, Oxman A, Vist G, Kunz R, Falck-Ytter Y, Alonso-Coello P . GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008; 336(7650):924-6. PMC: 2335261. DOI: 10.1136/bmj.39489.470347.AD. View

13.

Chiarotto A, Ostelo R, Boers M, Terwee C . A systematic review highlights the need to investigate the content validity of patient-reported outcome measures for physical functioning in patients with low back pain. J Clin Epidemiol. 2017; 95:73-93. DOI: 10.1016/j.jclinepi.2017.11.005. View

14.

Valderas J, Ferrer M, Mendivil J, Garin O, Rajmil L, Herdman M . Development of EMPRO: a tool for the standardized assessment of patient-reported outcome measures. Value Health. 2008; 11(4):700-8. DOI: 10.1111/j.1524-4733.2007.00309.x. View

15.

Prinsen C, Vohra S, Rose M, Boers M, Tugwell P, Clarke M . How to select outcome measurement instruments for outcomes included in a "Core Outcome Set" - a practical guideline. Trials. 2016; 17(1):449. PMC: 5020549. DOI: 10.1186/s13063-016-1555-2. View

16.

Vogt D, King D, King L . Focus groups in psychological assessment: enhancing content validity by consulting members of the target population. Psychol Assess. 2004; 16(3):231-43. DOI: 10.1037/1040-3590.16.3.231. View

17.

Lohr K, Aaronson N, Alonso J, Burnam M, Patrick D, PERRIN E . Evaluating quality-of-life and health status instruments: development of scientific review criteria. Clin Ther. 1996; 18(5):979-92. DOI: 10.1016/s0149-2918(96)80054-3. View

18.

Patrick D, Burke L, Gwaltney C, Leidy N, Martin M, Molsen E . Content validity--establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO good research practices task force report: part 1--eliciting concepts for a new PRO.... Value Health. 2011; 14(8):967-77. DOI: 10.1016/j.jval.2011.06.014. View

19.

Olshansky E, Lakes K, Vaughan J, Gravem D, Rich J, David M . Enhancing the Construct and Content Validity of Rating Scales for Clinical Research: Using Qualitative Methods to Develop a Rating Scale to Assess Parental Perceptions of Their Role in Promoting Infant Exercise. Int J Educ Psychol Assess. 2013; 10(1):36-50. PMC: 3806144. View

20.

Mokkink L, Terwee C, Knol D, Stratford P, Alonso J, Patrick D . The COSMIN checklist for evaluating the methodological quality of studies on measurement properties: a clarification of its content. BMC Med Res Methodol. 2010; 10:22. PMC: 2848183. DOI: 10.1186/1471-2288-10-22. View