A Comparison of Artificial Intelligence Versus Radiologists in the Diagnosis of Thyroid Nodules Using Ultrasonography: a Systematic Review and Meta-analysis

Overview

Journal Eur Arch Otorhinolaryngol

Specialty Otorhinolaryngology

Date 2022 Jun 29

PMID 35767056

Authors

Pimrada Potipimpanon

Natamon Charakorn

Prakobkiat Hirunwiwatkul

Affiliations

Soon will be listed here.

Abstract

Background: Thyroid nodules are common. Ultrasonography (US) is the first investigation for thyroid nodules. Artificial Intelligence (AI) is widely integrated into medical diagnosis to provide additional information. The primary objective of this study was to accumulate the pooled sensitivity and specificity between all available AI and radiologists using thyroid US imaging. The secondary objective was to compare AI's diagnostic performance to that of radiologists.

Materials And Methods: A systematic review meta-analysis. PubMed, Scopus, Web of Science, and Cochrane Library data were searched for studies from inception until June 11, 2020.

Results: Twenty five studies were included in this meta-analysis. The pooled sensitivity and specificity of AI were 0.86 (95% CI 0.81-0.91) and 0.78 (95% CI 0.73-0.83), respectively. The pooled sensitivity and specificity of radiologists were 0.85 (95% CI 0.80-0.89) and 0.82 (95% CI 0.77-0.86), respectively. The accuracy of AI and radiologists is equivalent in terms of AUC [AI 0.89 (95% CI 0.86-0.92), radiologist 0.91 (95% CI 0.88-0.93)]. The diagnostic odd ratio (DOR) between AI 23.10 (95% CI 14.20-37.58) and radiologists 27.12 (95% CI 17.45-42.16) had no statistically significant difference (P = 0.56). Meta-regression analysis revealed that Deep Learning AI had significantly greater sensitivity and specificity than classic machine learning AI (P < 0.001).

Conclusion: AI demonstrated comparable performance to radiologists in diagnosing benign and malignant thyroid nodules using ultrasonography. Additional research to establish its equivalency should be conducted.

Citing Articles

Thyroid nodules: diagnosis and management.

Grani G, Sponziello M, Filetti S, Durante C Nat Rev Endocrinol. 2024; 20(12):715-728.

PMID: 39152228 DOI: 10.1038/s41574-024-01025-4.

The Use of Artificial Intelligence in the Diagnosis and Classification of Thyroid Nodules: An Update.

Ludwig M, Ludwig B, Mikula A, Biernat S, Rudnicki J, Kaliszewski K Cancers (Basel). 2023; 15(3).

PMID: 36765671 PMC: 9913834. DOI: 10.3390/cancers15030708.

References

Xu L, Gao J, Wang Q, Yin J, Yu P, Bai B . Computer-Aided Diagnosis Systems in Diagnosing Malignant Thyroid Nodules on Ultrasonography: A Systematic Review and Meta-Analysis. Eur Thyroid J. 2020; 9(4):186-193. PMC: 7445671. DOI: 10.1159/000504390. View

Shin I, Kim Y, Han K, Lee E, Kim H, Shin J . Application of machine learning to ultrasound images to differentiate follicular neoplasms of the thyroid gland. Ultrasonography. 2020; 39(3):257-265. PMC: 7315296. DOI: 10.14366/usg.19069. View

Whiting P, Rutjes A, Westwood M, Mallett S, Deeks J, Reitsma J . QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med. 2011; 155(8):529-36. DOI: 10.7326/0003-4819-155-8-201110180-00009. View

Xia S, Yao J, Zhou W, Dong Y, Xu S, Zhou J . A computer-aided diagnosing system in the evaluation of thyroid nodules-experience in a specialized thyroid center. World J Surg Oncol. 2019; 17(1):210. PMC: 6898946. DOI: 10.1186/s12957-019-1752-z. View

Ko S, Lee J, Yoon J, Na H, Hong E, Han K . Deep convolutional neural network for the diagnosis of thyroid nodules on ultrasound. Head Neck. 2019; 41(4):885-891. DOI: 10.1002/hed.25415. View

Wu H, Deng Z, Zhang B, Liu Q, Chen J . Classifier Model Based on Machine Learning Algorithms: Application to Differential Diagnosis of Suspicious Thyroid Nodules via Sonography. AJR Am J Roentgenol. 2016; 207(4):859-864. DOI: 10.2214/AJR.15.15813. View

Fresilli D, Grani G, De Pascali M, Alagna G, Tassone E, Ramundo V . Computer-aided diagnostic system for thyroid nodule sonographic evaluation outperforms the specificity of less experienced examiners. J Ultrasound. 2020; 23(2):169-174. PMC: 7242558. DOI: 10.1007/s40477-020-00453-y. View

Buda M, Wildman-Tobriner B, Hoang J, Thayer D, Tessler F, Middleton W . Management of Thyroid Nodules Seen on US Images: Deep Learning May Match Performance of Radiologists. Radiology. 2019; 292(3):695-701. PMC: 10257128. DOI: 10.1148/radiol.2019181343. View

Park V, Han K, Seong Y, Park M, Kim E, Moon H . Diagnosis of Thyroid Nodules: Performance of a Deep Learning Convolutional Neural Network Model vs. Radiologists. Sci Rep. 2019; 9(1):17843. PMC: 6882804. DOI: 10.1038/s41598-019-54434-1. View

10.

Yoo Y, Ha E, Cho Y, Kim H, Han M, Kang S . Computer-Aided Diagnosis of Thyroid Nodules via Ultrasonography: Initial Clinical Experience. Korean J Radiol. 2018; 19(4):665-672. PMC: 6005935. DOI: 10.3348/kjr.2018.19.4.665. View

11.

Yang H, Lin J, Jin C, Meng L, Wu S, Chen Y . The Predictive Value of the Follicular Output Rate on Pregnancy Outcome of Patients with Polycystic Ovary Syndrome Undergoing In Vitro Fertilization and Embryo Transfer. Med Sci Monit. 2020; 26:e916175. PMC: 7737407. DOI: 10.12659/MSM.916175. View

12.

Barczynski M, Stopa-Barczynska M, Wojtczak B, Czarniecka A, Konturek A . Clinical validation of S-Detect mode in semi-automated ultrasound classification of thyroid lesions in surgical office. Gland Surg. 2020; 9(Suppl 2):S77-S85. PMC: 7044084. DOI: 10.21037/gs.2019.12.23. View

13.

Tangjaturonrasme N, Vatanasapt P, Bychkov A . Epidemiology of head and neck cancer in Thailand. Asia Pac J Clin Oncol. 2017; 14(1):16-22. DOI: 10.1111/ajco.12757. View

14.

Dean D, Gharib H . Epidemiology of thyroid nodules. Best Pract Res Clin Endocrinol Metab. 2008; 22(6):901-11. DOI: 10.1016/j.beem.2008.09.019. View

15.

Gao L, Liu R, Jiang Y, Song W, Wang Y, Liu J . Computer-aided system for diagnosing thyroid nodules on ultrasound: A comparison with radiologist-based clinical assessments. Head Neck. 2017; 40(4):778-783. DOI: 10.1002/hed.25049. View

16.

Gitto S, Grassi G, Angelis C, Monaco C, Sdao S, Sardanelli F . A computer-aided diagnosis system for the assessment and characterization of low-to-high suspicion thyroid nodules on ultrasound. Radiol Med. 2018; 124(2):118-125. DOI: 10.1007/s11547-018-0942-z. View

17.

Moon W, Baek J, Jung S, Kim D, Kim E, Kim J . Ultrasonography and the ultrasound-based management of thyroid nodules: consensus statement and recommendations. Korean J Radiol. 2011; 12(1):1-14. PMC: 3017873. DOI: 10.3348/kjr.2011.12.1.1. View

18.

Lee E, Ha H, Kim H, Moon H, Byon J, Huh S . Differentiation of thyroid nodules on US using features learned and extracted from various convolutional neural networks. Sci Rep. 2019; 9(1):19854. PMC: 6934479. DOI: 10.1038/s41598-019-56395-x. View

19.

Zhang B, Tian J, Pei S, Chen Y, He X, Dong Y . Machine Learning-Assisted System for Thyroid Nodule Diagnosis. Thyroid. 2019; 29(6):858-867. DOI: 10.1089/thy.2018.0380. View

20.

Reverter J, Vazquez F, Puig-Domingo M . Diagnostic Performance Evaluation of a Computer-Assisted Imaging Analysis System for Ultrasound Risk Stratification of Thyroid Nodules. AJR Am J Roentgenol. 2019; 213(1):169-174. DOI: 10.2214/AJR.18.20740. View