Automated Semantic Indexing of Figure Captions to Improve Radiology Image Retrieval

Overview

Journal J Am Med Inform Assoc

Publisher Oxford University Press

Specialty Medical Informatics

Date 2009 Mar 6

PMID 19261938

Citations 6

Authors

Charles E Kahn Jr

Daniel L Rubin

Affiliations

Soon will be listed here.

Abstract

Objective: We explored automated concept-based indexing of unstructured figure captions to improve retrieval of images from radiology journals.

Design: The MetaMap Transfer program (MMTx) was used to map the text of 84,846 figure captions from 9,004 peer-reviewed, English-language articles to concepts in three controlled vocabularies from the UMLS Metathesaurus, version 2006AA. Sampling procedures were used to estimate the standard information-retrieval metrics of precision and recall, and to evaluate the degree to which concept-based retrieval improved image retrieval.

Measurements: Precision was estimated based on a sample of 250 concepts. Recall was estimated based on a sample of 40 concepts. The authors measured the impact of concept-based retrieval to improve upon keyword-based retrieval in a random sample of 10,000 search queries issued by users of a radiology image search engine.

Results: Estimated precision was 0.897 (95% confidence interval, 0.857-0.937). Estimated recall was 0.930 (95% confidence interval, 0.838-1.000). In 5,535 of 10,000 search queries (55%), concept-based retrieval found results not identified by simple keyword matching; in 2,086 searches (21%), more than 75% of the results were found by concept-based search alone.

Conclusion: Concept-based indexing of radiology journal figure captions achieved very high precision and recall, and significantly improved image retrieval.

Citing Articles

Extracting Semantics from Census-based Reference Data.

Harris D, Seyedtalebi N Proc IEEE Int Conf Semant Comput. 2022; 2021:88-89.

PMID: 35252462 PMC: 8896120. DOI: 10.1109/icsc50631.2021.00022.

Semantic-Enhanced Query Expansion System for Retrieving Medical Image Notes.

Zhao Y, Fesharaki N, Li X, Patrick T, Luo J J Med Syst. 2018; 42(6):105.

PMID: 29696548 DOI: 10.1007/s10916-018-0954-1.

Neuroimaging Feature Terminology: A Controlled Terminology for the Annotation of Brain Imaging Features.

Iyappan A, Younesi E, Redolfi A, Vrooman H, Khanna S, Frisoni G J Alzheimers Dis. 2017; 59(4):1153-1169.

PMID: 28731430 PMC: 5611802. DOI: 10.3233/JAD-161148.

Imaging informatics: essential tools for the delivery of imaging services.

Mendelson D, Rubin D Acad Radiol. 2013; 20(10):1195-212.

PMID: 24029051 PMC: 4072254. DOI: 10.1016/j.acra.2013.07.006.

Improving image retrieval effectiveness via query expansion using MeSH hierarchical structure.

Crespo Azcarate M, Mata Vazquez J, Mana Lopez M J Am Med Inform Assoc. 2012; 20(6):1014-20.

PMID: 22952301 PMC: 3822102. DOI: 10.1136/amiajnl-2012-000943.

References

Jin Y, McDonald R, Lerman K, Mandel M, Carroll S, Liberman M . Automated recognition of malignancy mentions in biomedical literature. BMC Bioinformatics. 2006; 7:492. PMC: 1657036. DOI: 10.1186/1471-2105-7-492. View

Friedlin J, McDonald C . A natural language processing system to extract and code concepts relating to congestive heart failure from chest radiology reports. AMIA Annu Symp Proc. 2007; :269-73. PMC: 1839544. View

Uzuner O . Second i2b2 workshop on natural language processing challenges for clinical records. AMIA Annu Symp Proc. 2008; :1252-3. View

Rosse C, Mejino Jr J . A reference ontology for biomedical informatics: the Foundational Model of Anatomy. J Biomed Inform. 2004; 36(6):478-500. DOI: 10.1016/j.jbi.2003.11.007. View

Nadkarni P, Chen R, Brandt C . UMLS concept indexing for production databases: a feasibility study. J Am Med Inform Assoc. 2001; 8(1):80-91. PMC: 134593. DOI: 10.1136/jamia.2001.0080080. View

Hersh W, Hickam D, Haynes R, McKibbon K . A performance and failure analysis of SAPHIRE with a MEDLINE test collection. J Am Med Inform Assoc. 1994; 1(1):51-60. PMC: 116184. DOI: 10.1136/jamia.1994.95236136. View

Evans D, Hersh W, Monarch I, Lefferts R, Handerson S . Automatic indexing of abstracts via natural-language processing using a simple thesaurus. Med Decis Making. 1991; 11(4 Suppl):S108-15. View

Yu H . Towards answering biological questions with experimental evidence: automatically identifying text that summarize image content in full-text articles. AMIA Annu Symp Proc. 2007; :834-8. PMC: 1839512. View

Humphreys B, Lindberg D, Schoolman H, Barnett G . The Unified Medical Language System: an informatics research collaboration. J Am Med Inform Assoc. 1998; 5(1):1-11. PMC: 61271. DOI: 10.1136/jamia.1998.0050001. View

10.

Huang Y, Lowe H, Hersh W . A pilot study of contextual UMLS indexing to improve the precision of concept-based representation in XML-structured clinical radiology reports. J Am Med Inform Assoc. 2003; 10(6):580-7. PMC: 264436. DOI: 10.1197/jamia.M1369. View

11.

Rubin D . Creating and curating a terminology for radiology: ontology modeling and analysis. J Digit Imaging. 2007; 21(4):355-62. PMC: 3043845. DOI: 10.1007/s10278-007-9073-0. View

12.

Muller H, Kenny E, Sternberg P . Textpresso: an ontology-based information retrieval and extraction system for biological literature. PLoS Biol. 2004; 2(11):e309. PMC: 517822. DOI: 10.1371/journal.pbio.0020309. View

13.

Tuttle M, Olson N, Keck K, Cole W, Erlbaum M, Sherertz D . Metaphrase: an aid to the clinical conceptualization and formalization of patient problems in healthcare enterprises. Methods Inf Med. 1998; 37(4-5):373-83. View

14.

Miller R, Gieszczykiewicz F, Vries J, Cooper G . CHARTLINE: providing bibliographic references relevant to patient charts using the UMLS Metathesaurus Knowledge Sources. Proc Annu Symp Comput Appl Med Care. 1992; :86-90. PMC: 2248100. View

15.

Kahn Jr C, Thao C . GoldMiner: a radiology image search engine. AJR Am J Roentgenol. 2007; 188(6):1475-8. DOI: 10.2214/AJR.06.1740. View

16.

Mamlin B, Heinze D, McDonald C . Automated extraction and normalization of findings from cancer-related free-text radiology reports. AMIA Annu Symp Proc. 2004; :420-4. PMC: 1479955. View

17.

Aronson A . Effective mapping of biomedical text to the UMLS Metathesaurus: the MetaMap program. Proc AMIA Symp. 2002; :17-21. PMC: 2243666. View

18.

Bodenreider O . The Unified Medical Language System (UMLS): integrating biomedical terminology. Nucleic Acids Res. 2003; 32(Database issue):D267-70. PMC: 308795. DOI: 10.1093/nar/gkh061. View

19.

Langlotz C . RadLex: a new method for indexing online educational materials. Radiographics. 2006; 26(6):1595-7. DOI: 10.1148/rg.266065168. View

20.

Browne A, Divita G, Aronson A, McCray A . UMLS language and vocabulary tools. AMIA Annu Symp Proc. 2004; :798. PMC: 1480270. View