Validity and Reliability International Classification of Diseases-10 Codes for All Forms of Injury: A Systematic Review

Overview

Journal PLoS One

Specialties General Medicine
Science

Date 2024 Feb 29

PMID 38421992

Authors

Sarah Paleczny

Nosakhare Osagie

Jai Sethi

Michael Cusimano

Affiliations

Soon will be listed here.

Abstract

Background: Intentional and unintentional injuries are a leading cause of death and disability globally. International Classification of Diseases (ICD), Tenth Revision (ICD-10) codes are used to classify injuries in administrative health data and are widely used for health care planning and delivery, research, and policy. However, a systematic review of their overall validity and reliability has not yet been done.

Objective: To conduct a systematic review of the validity and reliability of external cause injury ICD-10 codes.

Methods: MEDLINE, EMBASE, COCHRANE, and SCOPUS were searched (inception to April 2023) for validity and/or reliability studies of ICD-10 external cause injury codes in all countries for all ages. We examined all available data for external cause injuries and injuries related to specific body regions. Validity was defined by sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). Reliability was defined by inter-rater reliability (IRR), measured by Krippendorff's alpha, Cohen's Kappa, and/or Fleiss' kappa.

Results: Twenty-seven published studies from 2006 to 2023 were included. Across all injuries, the mean outcome values and ranges were sensitivity: 61.6% (35.5%-96.0%), specificity: 91.6% (85.8%-100%), PPV: 74.9% (58.6%-96.5%), NPV: 80.2% (44.6%-94.4%), Cohen's kappa: 0.672 (0.480-0.928), Krippendorff's alpha: 0.453, and Fleiss' kappa: 0.630. Poisoning and hand and wrist injuries had higher mean sensitivity (84.4% and 96.0%, respectively), while self-harm and spinal cord injuries were lower (35.5% and 36.4%, respectively). Transport and pedestrian injuries and hand and wrist injuries had high PPVs (96.5% and 92.0%, respectively). Specificity and NPV were generally high, except for abuse (NPV 44.6%).

Conclusions And Significance: The validity and reliability of ICD-10 external cause injury codes vary based on the injury types coded and the outcomes examined, and overall, they only perform moderately well. Future work, potentially utilizing artificial intelligence, may improve the validity and reliability of ICD codes used to document injuries.

Citing Articles

Determinant Factors and Characteristics of Injuries Among Preschool Children in Public Daycare Centers.

Kouli O, Morela E, Papanikolaou E, Dalakis A, Karageorgopoulou M, Kambas A Children (Basel). 2025; 12(2).

PMID: 40003353 PMC: 11854655. DOI: 10.3390/children12020251.

Correction: Validity and reliability International Classification of Diseases-10 codes for all forms of injury: A systematic review.

PLoS One. 2025; 20(1):e0318388.

PMID: 39854550 PMC: 11761078. DOI: 10.1371/journal.pone.0318388.

Memory Loss and Missteps: Investigating Fall Risks in Alzheimer's and Dementia Patients.

Namoos A, Thomson N, Bradley S, Rudderman A, Aboutanos M Adv Geriatr Med Res. 2024; 6(3).

PMID: 39421020 PMC: 11485985. DOI: 10.20900/agmr20240005.

Adoption of network and plan-do-check-action in the international classification of disease 10 coding.

Lian B World J Clin Cases. 2024; 12(19):3734-3743.

PMID: 38994277 PMC: 11235436. DOI: 10.12998/wjcc.v12.i19.3734.

Parkinson's Disease Progression and Exposure to Contaminated Water at Camp Lejeune.

Goldman S, Weaver F, Gonzalez B, Stroupe K, Cao L, Colletta K Mov Disord. 2024; 39(10):1732-1739.

PMID: 38988230 PMC: 11490380. DOI: 10.1002/mds.29922.

References

Shehab N, Ziemba R, Campbell K, Geller A, Moro R, Gage B . Assessment of ICD-10-CM code assignment validity for case finding of outpatient anticoagulant-related bleeding among Medicare beneficiaries. Pharmacoepidemiol Drug Saf. 2019; 28(7):951-964. DOI: 10.1002/pds.4783. View

McHugh M . Interrater reliability: the kappa statistic. Biochem Med (Zagreb). 2012; 22(3):276-82. PMC: 3900052. View

McKenzie K, Enraght-Moony E, Walker S, McClure R, Harrison J . Accuracy of external cause-of-injury coding in hospital records. Inj Prev. 2009; 15(1):60-4. DOI: 10.1136/ip.2008.019935. View

Karkhaneh M, Hagel B, Couperthwaite A, Saunders L, Voaklander D, Rowe B . Emergency department coding of bicycle and pedestrian injuries during the transition from ICD-9 to ICD-10. Inj Prev. 2011; 18(2):88-93. PMC: 3313444. DOI: 10.1136/ip.2010.031302. View

Randall J, Roos L, Lix L, Katz L, Bolton J . Emergency department and inpatient coding for self-harm and suicide attempts: Validation using clinician assessment data. Int J Methods Psychiatr Res. 2017; 26(3). PMC: 6877200. DOI: 10.1002/mpr.1559. View

McCormick N, Bhole V, Lacaille D, Avina-Zubieta J . Validity of Diagnostic Codes for Acute Stroke in Administrative Databases: A Systematic Review. PLoS One. 2015; 10(8):e0135834. PMC: 4546158. DOI: 10.1371/journal.pone.0135834. View

McCormick N, Lacaille D, Bhole V, Avina-Zubieta J . Validity of myocardial infarction diagnoses in administrative databases: a systematic review. PLoS One. 2014; 9(3):e92286. PMC: 3969323. DOI: 10.1371/journal.pone.0092286. View

Henderson T, Shepheard J, Sundararajan V . Quality of diagnosis and procedure coding in ICD-10 administrative data. Med Care. 2006; 44(11):1011-9. DOI: 10.1097/01.mlr.0000228018.48783.34. View

Hagen E, Rekand T, Gilhus N, Gronning M . Diagnostic coding accuracy for traumatic spinal cord injuries. Spinal Cord. 2008; 47(5):367-71. DOI: 10.1038/sc.2008.118. View

10.

Peterson A, Gabella B, Johnson J, Hume B, Liu A, Costich J . Multisite medical record review of emergency department visits for unspecified injury of head following the ICD-10-CM coding transition. Inj Prev. 2021; 27(S1):i13-i18. PMC: 7948189. DOI: 10.1136/injuryprev-2019-043517. View

11.

Furlan J, Fehlings M . The National Trauma Registry as a Canadian spine trauma database: a validation study using an institutional clinical database. Neuroepidemiology. 2011; 37(2):96-101. DOI: 10.1159/000330835. View

12.

Chiang M, Shao S, Liao S . Validation of Diagnostic Codes to Identify Carbon Monoxide Poisoning in Taiwan's Claims Data. Front Pharmacol. 2022; 13:882632. PMC: 9240270. DOI: 10.3389/fphar.2022.882632. View

13.

Paydar S, Asadi F . Evaluating the effect of an in-service training workshop on ICD-10 coding instructions of pregnancy, childbirth and the puerperium for clinical coders. J Med Life. 2021; 14(4):565-569. PMC: 8485367. DOI: 10.25122/jml-2021-0062. View

14.

Welk B, Loh E, Shariff S, Liu K, Siddiqi F . An administrative data algorithm to identify traumatic spinal cord injured patients: a validation study. Spinal Cord. 2013; 52(1):34-8. DOI: 10.1038/sc.2013.134. View

15.

Watzlaf V, Garvin J, Moeini S, Anania-Firouzan P . The effectiveness of ICD-10-CM in capturing public health diseases. Perspect Health Inf Manag. 2007; 4:6. PMC: 2047296. View

16.

Hunt P, Hackman H, Berenholz G, McKeown L, Davis L, Ozonoff V . Completeness and accuracy of International Classification of Disease (ICD) external cause of injury codes in emergency department electronic data. Inj Prev. 2007; 13(6):422-5. PMC: 2598290. DOI: 10.1136/ip.2007.015859. View

17.

Page M, McKenzie J, Bossuyt P, Boutron I, Hoffmann T, Mulrow C . The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021; 372:n71. PMC: 8005924. DOI: 10.1136/bmj.n71. View

18.

Jeong G . Artificial intelligence, machine learning, and deep learning in women's health nursing. Korean J Women Health Nurs. 2022; 26(1):5-9. PMC: 9334197. DOI: 10.4069/kjwhn.2020.03.11. View

19.

Trinh L, Achat H, Loh S, Pascoe R, Assareh H, Stubbs J . Validity of routinely collected data in identifying hip fractures at a major tertiary hospital in Australia. Health Inf Manag. 2017; 47(1):38-45. DOI: 10.1177/1833358317721305. View

20.

Chen P, Wang S, Liao W, Kuo L, Chen K, Lin Y . Automatic ICD-10 Coding and Training System: Deep Neural Network Based on Supervised Learning. JMIR Med Inform. 2021; 9(8):e23230. PMC: 8441604. DOI: 10.2196/23230. View