Assessment Fidelity in Aphasia Research

Overview

Journal Am J Speech Lang Pathol

Specialty Otorhinolaryngology

Date 2016 Dec 21

PMID 27997953

Citations 8

Authors

Jessica D Richardson

Sarah Grace Hudspeth Dalton

Jennifer Shafer

Janet Patterson

Affiliations

Soon will be listed here.

Abstract

Purpose: In aphasia treatment literature, scarce attention is paid to factors that may reduce a study's validity, including adherence to assessment and treatment procedures (i.e., fidelity). Although guidelines have been established for evaluating and reporting treatment fidelity, none exist for assessment fidelity.

Method: We reviewed treatment fidelity guidelines and related literature to identify assessment fidelity components. We then examined 88 aphasia treatment studies published between 2010 and 2015 and report the frequency with which researchers provide information regarding the following assessment fidelity components: assessment instruments, assessor qualifications, assessor or rater training, assessment delivery, assessor or rater reliability, and assessor blinding.

Results: We found that 4.5% of studies reported information regarding assessment instruments, 35.2% reported information regarding assessor qualifications, 6.85% reported information regarding assessor or rater training, 37.5% reported information regarding assessor or rater reliability, 27.3% reported on assessor blinding, and no studies reported information regarding assessment delivery.

Conclusions: There is a paucity of assessment fidelity information reported in aphasia treatment research. The authors propose a set of guidelines to ensure readers will be able to evaluate assessment fidelity, and thus study validity.

Citing Articles

Standardised data collection from people with dementia over the telephone: A qualitative study of the experience of DETERMIND programme researchers in a pandemic.

Gridley K, Dixon J, Hicks B, Birks Y, Baxter K, Miles E Dementia (London). 2023; 22(8):1718-1737.

PMID: 37495232 PMC: 10372513. DOI: 10.1177/14713012231190585.

Assessment Fidelity of Parents Implementing a Standardized Telehealth Infant Autism Screener.

Phillips A, Campi E, Talbott M, Baranek G OTJR (Thorofare N J). 2023; 43(3):360-367.

PMID: 37089013 PMC: 10330541. DOI: 10.1177/15394492231164943.

Measuring communication as a core outcome in aphasia trials: Results of the ROMA-2 international core outcome set development meeting.

Wallace S, Worrall L, Rose T, Alyahya R, Babbitt E, Beeke S Int J Lang Commun Disord. 2022; 58(4):1017-1028.

PMID: 36583427 PMC: 10946976. DOI: 10.1111/1460-6984.12840.

Assessment of language impairment and function.

Richardson J, Dalton S Handb Clin Neurol. 2022; 185:175-193.

PMID: 35078598 PMC: 11363586. DOI: 10.1016/B978-0-12-823384-9.00009-8.

Standardizing Assessment of Spoken Discourse in Aphasia: A Working Group With Deliverables.

Stark B, Dutta M, Murray L, Bryant L, Fromm D, MacWhinney B Am J Speech Lang Pathol. 2020; 30(1S):491-502.

PMID: 32585117 PMC: 9128722. DOI: 10.1044/2020_AJSLP-19-00093.

References

Gagnier J, Kienle G, Altman D, Moher D, Sox H, Riley D . The CARE Guidelines: Consensus-based Clinical Case Reporting Guideline Development. Glob Adv Health Med. 2014; 2(5):38-43. PMC: 3833570. DOI: 10.7453/gahmj.2013.008. View

Cummings K, Biancarosa G, Schaper A, Reed D . Examiner error in curriculum-based measurement of oral reading. J Sch Psychol. 2014; 52(4):361-75. DOI: 10.1016/j.jsp.2014.05.007. View

Jacobson N, Truax P . Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. J Consult Clin Psychol. 1991; 59(1):12-9. DOI: 10.1037//0022-006x.59.1.12. View

MacWhinney B, Fromm D, Forbes M, Holland A . AphasiaBank: Methods for Studying Discourse. Aphasiology. 2012; 25(11):1286-1307. PMC: 3424615. DOI: 10.1080/02687038.2011.589893. View

Holland A, Fromm D, DeRuyter F, Stein M . Treatment efficacy: aphasia. J Speech Hear Res. 1996; 39(5):S27-36. DOI: 10.1044/jshr.3905.s27. View

Dusenbury L, Brannigan R, Falco M, Hansen W . A review of research on fidelity of implementation: implications for drug abuse prevention in school settings. Health Educ Res. 2003; 18(2):237-56. DOI: 10.1093/her/18.2.237. View

Poolman R, Struijs P, Krips R, Sierevelt I, Marti R, Farrokhyar F . Reporting of outcomes in orthopaedic randomized trials: does blinding of outcome assessors matter?. J Bone Joint Surg Am. 2007; 89(3):550-8. DOI: 10.2106/JBJS.F.00683. View

Schulz K, Chalmers I, Altman D . The landscape and lexicon of blinding in randomized trials. Ann Intern Med. 2002; 136(3):254-9. DOI: 10.7326/0003-4819-136-3-200202050-00022. View

Borrelli B . The Assessment, Monitoring, and Enhancement of Treatment Fidelity In Public Health Clinical Trials. J Public Health Dent. 2011; 71(s1):S52-S63. PMC: 3074245. DOI: 10.1111/j.1752-7325.2011.00233.x. View

10.

Salman R, Beller E, Kagan J, Hemminki E, Phillips R, Savulescu J . Increasing value and reducing waste in biomedical research regulation and management. Lancet. 2014; 383(9912):176-85. PMC: 3952153. DOI: 10.1016/S0140-6736(13)62297-7. View

11.

Chan A, Tetzlaff J, Altman D, Laupacis A, Gotzsche P, Krleza-Jeric K . SPIRIT 2013 statement: defining standard protocol items for clinical trials. Ann Intern Med. 2013; 158(3):200-7. PMC: 5114123. DOI: 10.7326/0003-4819-158-3-201302050-00583. View

12.

Bothe A, Richardson J . Statistical, practical, clinical, and personal significance: definitions and applications in speech-language pathology. Am J Speech Lang Pathol. 2011; 20(3):233-42. DOI: 10.1044/1058-0360(2011/10-0034). View

13.

Robey R . The efficacy of treatment for aphasic persons: a meta-analysis. Brain Lang. 1994; 47(4):582-608. DOI: 10.1006/brln.1994.1060. View

14.

Pildal J, Hrobjartsson A, Jorgensen K, Hilden J, Altman D, Gotzsche P . Impact of allocation concealment on conclusions drawn from meta-analyses of randomized trials. Int J Epidemiol. 2007; 36(4):847-57. DOI: 10.1093/ije/dym087. View

15.

Moher D, Liberati A, Tetzlaff J, Altman D . Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement. Open Med. 2011; 3(3):e123-30. PMC: 3090117. View

16.

Kelly H, Brady M, Enderby P . Speech and language therapy for aphasia following stroke. Cochrane Database Syst Rev. 2010; (5):CD000425. DOI: 10.1002/14651858.CD000425.pub2. View

17.

Hoffmann T, Glasziou P, Boutron I, Milne R, Perera R, Moher D . Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ. 2014; 348:g1687. DOI: 10.1136/bmj.g1687. View

18.

Ioannidis J, Greenland S, Hlatky M, Khoury M, Macleod M, Moher D . Increasing value and reducing waste in research design, conduct, and analysis. Lancet. 2014; 383(9912):166-75. PMC: 4697939. DOI: 10.1016/S0140-6736(13)62227-8. View

19.

Durlak J, DuPre E . Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. Am J Community Psychol. 2008; 41(3-4):327-50. DOI: 10.1007/s10464-008-9165-0. View

20.

Hart T, Bagiella E . Design and implementation of clinical trials in rehabilitation research. Arch Phys Med Rehabil. 2012; 93(8 Suppl):S117-26. DOI: 10.1016/j.apmr.2011.11.039. View