Item Response Theory Modeling of the Verb Naming Test

Overview

Journal J Speech Lang Hear Res

Date 2023 Mar 31

PMID 37000934

Authors

Gerasimos Fergadiotis

Marianne Casilio

Michael Walsh Dickey

Stacey Steel

Hannele Nicholson

Mikala Fleegle

Alexander Swiderski

William D Hula

Affiliations

Soon will be listed here.

Abstract

Purpose: Item response theory (IRT) is a modern psychometric framework with several advantageous properties as compared with classical test theory. IRT has been successfully used to model performance on anomia tests in individuals with aphasia; however, all efforts to date have focused on noun production accuracy. The purpose of this study is to evaluate whether the Verb Naming Test (VNT), a prominent test of action naming, can be successfully modeled under IRT and evaluate its reliability.

Method: We used responses on the VNT from 107 individuals with chronic aphasia from AphasiaBank. Unidimensionality and local independence, two assumptions prerequisite to IRT modeling, were evaluated using factor analysis and Yen's statistic (Yen, 1984), respectively. The assumption of equal discrimination among test items was evaluated statistically via nested model comparisons and practically by using correlations of resulting IRT-derived scores. Finally, internal consistency, marginal and empirical reliability, and conditional reliability were evaluated.

Results: The VNT was found to be sufficiently unidimensional with the majority of item pairs demonstrating adequate local independence. An IRT model in which item discriminations are constrained to be equal demonstrated fit equivalent to a model in which unique discrimination parameters were estimated for each item. All forms of reliability were strong across the majority of IRT ability estimates.

Conclusions: Modeling the VNT using IRT is feasible, yielding ability estimates that are both informative and reliable. Future efforts are needed to quantify the validity of the VNT under IRT and determine the extent to which it measures the same construct as other anomia tests.

Supplemental Material: https://doi.org/10.23641/asha.22329235.

References

Li C . Confirmatory factor analysis with ordinal data: Comparing robust maximum likelihood and diagonally weighted least squares. Behav Res Methods. 2015; 48(3):936-49. DOI: 10.3758/s13428-015-0619-7. View

Fergadiotis G, Swiderski A, Hula W . Predicting confrontation naming item difficulty. Aphasiology. 2019; 33(6):689-709. PMC: 6713450. DOI: 10.1080/02687038.2018.1495310. View

Dell G, Martin N, Schwartz M . A Case-Series Test of the Interactive Two-step Model of Lexical Access: Predicting Word Repetition from Picture Naming. J Mem Lang. 2010; 56(4):490-520. PMC: 2981040. DOI: 10.1016/j.jml.2006.05.007. View

Szekely A, DAmico S, Devescovi A, Federmeier K, Herron D, Iyer G . Timed action and object naming. Cortex. 2005; 41(1):7-25. DOI: 10.1016/s0010-9452(08)70174-6. View

Kelley K, Pornprasertmanit S . Confidence intervals for population reliability coefficients: Evaluation of methods, recommendations, and software for composite measures. Psychol Methods. 2016; 21(1):69-92. DOI: 10.1037/a0040086. View

Willmes K . A new look at the Token Test using probabilistic test models. Neuropsychologia. 1981; 19(5):631-46. DOI: 10.1016/0028-3932(81)90001-4. View

Hula W, Kellough S, Fergadiotis G . Development and Simulation Testing of a Computerized Adaptive Version of the Philadelphia Naming Test. J Speech Lang Hear Res. 2015; 58(3):878-890. DOI: 10.1044/2015_JSLHR-L-14-0297. View

Fergadiotis G, Casilio M, Hula W, Swiderski A . Computer Adaptive Testing for the Assessment of Anomia Severity. Semin Speech Lang. 2021; 42(3):180-191. DOI: 10.1055/s-0041-1727252. View

Gordon J, Dell G . Learning to divide the labor between syntax and semantics: a connectionist account of deficits in light and heavy verb production. Brain Cogn. 2002; 48(2-3):376-81. View

10.

Dell G, SCHWARTZ M, Martin N, Saffran E, Gagnon D . Lexical access in aphasic and nonaphasic speakers. Psychol Rev. 1997; 104(4):801-38. DOI: 10.1037/0033-295x.104.4.801. View

11.

Bates E, Chen S, Tzeng O, Li P, Opie M . The noun-verb problem in Chinese aphasia. Brain Lang. 1991; 41(2):203-33. DOI: 10.1016/0093-934x(91)90153-r. View

12.

Fergadiotis G, Kellough S, Hula W . Item Response Theory Modeling of the Philadelphia Naming Test. J Speech Lang Hear Res. 2015; 58(3):865-877. DOI: 10.1044/2015_JSLHR-L-14-0249. View

13.

Levelt W, Roelofs A, Meyer A . A theory of lexical access in speech production. Behav Brain Sci. 2001; 22(1):1-38; discussion 38-75. DOI: 10.1017/s0140525x99001776. View

14.

Walker G, Basilakos A, Fridriksson J, Hickok G . Beyond Percent Correct: Measuring Change in Individual Picture Naming Ability. J Speech Lang Hear Res. 2021; 65(1):215-237. PMC: 9154021. DOI: 10.1044/2021_JSLHR-20-00205. View

15.

Thompson C, Shapiro L . Treating agrammatic aphasia within a linguistic framework: Treatment of Underlying Forms. Aphasiology. 2007; 19(10-11):1021-1036. PMC: 1847567. DOI: 10.1080/02687030544000227. View

16.

Berndt R, Haendiges A, Mitchum C, SANDSON J . Verb retrieval in aphasia. 2. Relationship to sentence processing. Brain Lang. 1997; 56(1):107-37. DOI: 10.1006/brln.1997.1728. View

17.

MacWhinney B, Fromm D, Forbes M, Holland A . AphasiaBank: Methods for Studying Discourse. Aphasiology. 2012; 25(11):1286-1307. PMC: 3424615. DOI: 10.1080/02687038.2011.589893. View

18.

Cherney L, Babbitt E, Semik P, Heinemann A . Psychometric properties of the communication Confidence Rating Scale for Aphasia (CCRSA): phase 1. Top Stroke Rehabil. 2011; 18(4):352-60. DOI: 10.1310/tsr1804-352. View

19.

Bentler P . Comparative fit indexes in structural models. Psychol Bull. 1990; 107(2):238-46. DOI: 10.1037/0033-2909.107.2.238. View

20.

Segalowitz S, Lane K . Lexical access of function versus content words. Brain Lang. 2000; 75(3):376-89. DOI: 10.1006/brln.2000.2361. View