Assessing ChatGPT's Summarization of Ga PSMA PET/CT Reports for Patients

Overview

Journal Abdom Radiol (NY)

Publisher Springer

Specialties Gastroenterology
Radiology

Date 2024 Sep 30

PMID 39347975

Authors

Ogun Bulbul

Hande Melike Bulbul

Esat Kaba

Affiliations

Soon will be listed here.

Abstract

Purpose: ChatGPT has recently been the subject of many studies, and its responses to medical questions have been successful. We examined ChatGPT-4's evaluation of structured Ga prostate-specific membrane antigen (PSMA) PET/CT reports of newly diagnosed prostate cancer patients.

Methods: Ga PSMA PET/CT reports of 164 patients were entered to ChatGPT-4. ChatGPT-4 was asked to respond the following questions according to the PET/CT reports: 1-Has the cancer in the prostate extended to organs adjacent to the prostate? 2-Has the cancer in the prostate spread to neighboring lymph nodes? 3-Has the cancer in the prostate spread to lymph nodes in distant areas? 4-Has the cancer in the prostate spread to the bones? 5-Has the cancer in the prostate spread to other organs? ChatGPT-4's responses were scored on a Likert-type scale for clarity and accuracy.

Results: The mean scores for clarity were 4.93 ± 0.32, 4.95 ± 0.25, 4.96 ± 0.19, 4.99 ± 0.11, and 4.96 ± 0.30, respectively. The mean scores for accuracy were 4.87 ∓ 0.61, 4.87 ∓ 0.62, 4.79 ± 0.83, 4.96 ± 0.25, and 4.93 ± 0.45, respectively. Patients with distant lymphatic metastases had a lower mean accuracy score than those without (4.28 ± 1.45 vs. 4.94 ± 0.39; p < 0.001). ChatGPT-4's responses in 13 patients (8%) had the potential for harmful information.

Conclusion: ChatGPT-4 successfully interprets structured Ga PSMA PET/CT reports of reports of newly diagnosed prostate cancer patients. However, it is unlikely that ChatGPT-4 evaluations will replace physicians' evaluations today, especially since it can produce fabricated information.

References

Stengel F, Stienen M, Ivanov M, Gandia-Gonzalez M, Raffa G, Ganau M . Can AI pass the written European Board Examination in Neurological Surgery? - Ethical and practical issues. Brain Spine. 2024; 4:102765. PMC: 10951784. DOI: 10.1016/j.bas.2024.102765. View

Chen T, Multala E, Kearns P, Delashaw J, Dumont A, Maraganore D . Assessment of ChatGPT's performance on neurology written board examination questions. BMJ Neurol Open. 2023; 5(2):e000530. PMC: 10626870. DOI: 10.1136/bmjno-2023-000530. View

Ghanem D, Nassar J, El Bachour J, Hanna T . ChatGPT Earns American Board Certification in Hand Surgery. Hand Surg Rehabil. 2024; 43(3):101688. DOI: 10.1016/j.hansur.2024.101688. View

Mallio C, Bernetti C, Sertorio A, Zobel B . ChatGPT in radiology structured reporting: analysis of ChatGPT-3.5 Turbo and GPT-4 in reducing word count and recalling findings. Quant Imaging Med Surg. 2024; 14(2):2096-2102. PMC: 10895108. DOI: 10.21037/qims-23-1300. View

Mallio C, Sertorio A, Bernetti C, Zobel B . Large language models for structured reporting in radiology: performance of GPT-4, ChatGPT-3.5, Perplexity and Bing. Radiol Med. 2023; 128(7):808-812. DOI: 10.1007/s11547-023-01651-4. View

Li H, Moon J, Iyer D, Balthazar P, Krupinski E, Bercu Z . Decoding radiology reports: Potential application of OpenAI ChatGPT to enhance patient understanding of diagnostic reports. Clin Imaging. 2023; 101:137-141. DOI: 10.1016/j.clinimag.2023.06.008. View

Schmidt S, Zimmerer A, Cucos T, Feucht M, Navas L . Simplifying radiologic reports with natural language processing: a novel approach using ChatGPT in enhancing patient understanding of MRI results. Arch Orthop Trauma Surg. 2023; 144(2):611-618. DOI: 10.1007/s00402-023-05113-4. View

Sarangi P, Lumbani A, Swarup M, Panda S, Sahoo S, Hui P . Assessing ChatGPT's Proficiency in Simplifying Radiological Reports for Healthcare Professionals and Patients. Cureus. 2024; 15(12):e50881. PMC: 10799309. DOI: 10.7759/cureus.50881. View

Lyu Q, Tan J, Zapadka M, Ponnatapura J, Niu C, Myers K . Translating radiology reports into plain language using ChatGPT and GPT-4 with prompt learning: results, limitations, and potential. Vis Comput Ind Biomed Art. 2023; 6(1):9. PMC: 10192466. DOI: 10.1186/s42492-023-00136-5. View

10.

Murthy V, Sonni I, Jariwala N, Juarez R, Reiter R, Raman S . The Role of PSMA PET/CT and PET/MRI in the Initial Staging of Prostate Cancer. Eur Urol Focus. 2021; 7(2):258-266. DOI: 10.1016/j.euf.2021.01.016. View

11.

Combes A, Palma C, Calopedos R, Wen L, Woo H, Fulham M . PSMA PET-CT in the Diagnosis and Staging of Prostate Cancer. Diagnostics (Basel). 2022; 12(11). PMC: 9689635. DOI: 10.3390/diagnostics12112594. View

12.

Yilmaz B, Turkay R, Colakoglu Y, Baytekin H, Ergul N, Sahin S . Comparison of preoperative locoregional Ga-68 PSMA-11 PET-CT and mp-MRI results with postoperative histopathology of prostate cancer. Prostate. 2019; 79(9):1007-1017. DOI: 10.1002/pros.23812. View

13.

Arslan A, Karaarslan E, Guner A, Saglican Y, Tuna M, Kural A . Comparing the Diagnostic Performance of Multiparametric Prostate MRI Versus 68Ga-PSMA PET-CT in the Evaluation Lymph Node Involvement and Extraprostatic Extension. Acad Radiol. 2020; 29(5):698-704. DOI: 10.1016/j.acra.2020.07.011. View

14.

McGowan A, Gui Y, Dobbs M, Shuster S, Cotter M, Selloni A . ChatGPT and Bard exhibit spontaneous citation fabrication during psychiatry literature search. Psychiatry Res. 2023; 326:115334. PMC: 10424704. DOI: 10.1016/j.psychres.2023.115334. View

15.

Blum M . ChatGPT Produces Fabricated References and Falsehoods When Used for Scientific Literature Search. J Card Fail. 2023; 29(9):1332-1334. DOI: 10.1016/j.cardfail.2023.06.015. View

16.

Bhattacharyya M, Miller V, Bhattacharyya D, Miller L . High Rates of Fabricated and Inaccurate References in ChatGPT-Generated Medical Content. Cureus. 2023; 15(5):e39238. PMC: 10277170. DOI: 10.7759/cureus.39238. View

17.

Brewer-Hofmann A, Sajjad S, Bekheet Z, Moy M, Wong T . Factors influencing patient understanding of information on radiology examinations. Skeletal Radiol. 2023; 52(8):1503-1509. PMC: 9933798. DOI: 10.1007/s00256-023-04301-y. View