Checklist for Evaluation of Image-Based Artificial Intelligence Reports in Dermatology: CLEAR Derm Consensus Guidelines From the International Skin Imaging Collaboration Artificial Intelligence Working Group

Overview

Journal JAMA Dermatol

Publisher American Medical Association

Specialty Dermatology

Date 2021 Dec 1

PMID 34851366

Citations 43

Authors

Roxana Daneshjou

Catarina Barata

Brigid Betz-Stablein

M Emre Celebi

Noel Codella

Marc Combalia

Pascale Guitera

David Gutman

Allan Halpern

Brian Helba

Harald Kittler

Kivanc Kose

Konstantinos Liopyris

Josep Malvehy

Han Seung Seog

H Peter Soyer

Eric R Tkaczyk

Philipp Tschandl

Veronica Rotemberg

Affiliations

Soon will be listed here.

Abstract

Importance: The use of artificial intelligence (AI) is accelerating in all aspects of medicine and has the potential to transform clinical care and dermatology workflows. However, to develop image-based algorithms for dermatology applications, comprehensive criteria establishing development and performance evaluation standards are required to ensure product fairness, reliability, and safety.

Objective: To consolidate limited existing literature with expert opinion to guide developers and reviewers of dermatology AI.

Evidence Review: In this consensus statement, the 19 members of the International Skin Imaging Collaboration AI working group volunteered to provide a consensus statement. A systematic PubMed search was performed of English-language articles published between December 1, 2008, and August 24, 2021, for "artificial intelligence" and "reporting guidelines," as well as other pertinent studies identified by the expert panel. Factors that were viewed as critical to AI development and performance evaluation were included and underwent 2 rounds of electronic discussion to achieve consensus.

Findings: A checklist of items was developed that outlines best practices of image-based AI development and assessment in dermatology.

Conclusions And Relevance: Clinically effective AI needs to be fair, reliable, and safe; this checklist of best practices will help both developers and reviewers achieve this goal.

Citing Articles

Policy Challenges in Ultra-Rare Cancers: Ethical, Social, and Legal Implications of Melanoma Prevention and Diagnosis in Children, Adolescents, and Young Adults.

Refolo P, Raimondi C, Battaglia L, Borras J, Closa P, Lo Scalzo A Healthcare (Basel). 2025; 13(3).

PMID: 39942510 PMC: 11818049. DOI: 10.3390/healthcare13030321.

AI for IMPACTS Framework for Evaluating the Long-Term Real-World Impacts of AI-Powered Clinician Tools: Systematic Review and Narrative Synthesis.

Jacob C, Brasier N, Laurenzi E, Heuss S, Mougiakakou S, Coltekin A J Med Internet Res. 2025; 27:e67485.

PMID: 39909417 PMC: 11840377. DOI: 10.2196/67485.

Diagnostic performance of generative pretrained transformer -4 with vision technology versus board-certified dermatologists: A comparative analysis using dermoscopic and clinical images.

Block B, Powers C, Chang A, Campbell C, Piontkowski A, Orloff J JAAD Int. 2025; 18():142-144.

PMID: 39790862 PMC: 11713480. DOI: 10.1016/j.jdin.2024.10.006.

Artificial Intelligence in the Non-Invasive Detection of Melanoma.

Mendi B, Kose K, Fleshner L, Adam R, Safai B, Farabi B Life (Basel). 2025; 14(12).

PMID: 39768310 PMC: 11678477. DOI: 10.3390/life14121602.

DERM12345: A Large, Multisource Dermatoscopic Skin Lesion Dataset with 40 Subclasses.

Yilmaz A, Yasar S, Gencoglan G, Temelkuran B Sci Data. 2024; 11(1):1302.

PMID: 39609462 PMC: 11604664. DOI: 10.1038/s41597-024-04104-3.

References

Roy A, Ren J, Azizi S, Loh A, Natarajan V, Mustafa B . Does your dermatology classifier know what it doesn't know? Detecting the long-tail of unseen conditions. Med Image Anal. 2021; 75:102274. DOI: 10.1016/j.media.2021.102274. View

Kelly B, Judge C, Bollard S, Clifford S, Healy G, Yeom K . Radiology artificial intelligence, a systematic evaluation of methods (RAISE): a systematic review protocol. Insights Imaging. 2020; 11(1):133. PMC: 7726044. DOI: 10.1186/s13244-020-00929-9. View

Elder D, Piepkorn M, Barnhill R, Longton G, Nelson H, Knezevich S . Pathologist characteristics associated with accuracy and reproducibility of melanocytic skin lesion interpretation. J Am Acad Dermatol. 2018; 79(1):52-59.e5. PMC: 6016831. DOI: 10.1016/j.jaad.2018.02.070. View

Wawira Gichoya J, McCoy L, Celi L, Ghassemi M . Equity in essence: a call for operationalising fairness in machine learning for healthcare. BMJ Health Care Inform. 2021; 28(1). PMC: 8733939. DOI: 10.1136/bmjhci-2020-100289. View

Campbell J, Lee A, Abramoff M, Keane P, Ting D, Lum F . Reporting Guidelines for Artificial Intelligence in Medical Research. Ophthalmology. 2020; 127(12):1596-1599. PMC: 7875521. DOI: 10.1016/j.ophtha.2020.09.009. View

Shelmerdine S, Arthurs O, Denniston A, Sebire N . Review of study reporting guidelines for clinical studies using artificial intelligence in healthcare. BMJ Health Care Inform. 2021; 28(1). PMC: 8383863. DOI: 10.1136/bmjhci-2021-100385. View

Chiang S, Picard R, Chiong W, Moss R, Worrell G, Rao V . Guidelines for Conducting Ethical Artificial Intelligence Research in Neurology: A Systematic Approach for Clinicians and Researchers. Neurology. 2021; 97(13):632-640. PMC: 8480407. DOI: 10.1212/WNL.0000000000012570. View

Du-Harpur X, Arthurs C, Ganier C, Woolf R, Laftah Z, Lakhan M . Clinically Relevant Vulnerabilities of Deep Machine Learning Systems for Skin Cancer Diagnosis. J Invest Dermatol. 2020; 141(4):916-920. PMC: 7990050. DOI: 10.1016/j.jid.2020.07.034. View

Pfau M, Walther G, von der Emde L, Berens P, Faes L, Fleckenstein M . [Artificial intelligence in ophthalmology : Guidelines for physicians for the critical evaluation of studies]. Ophthalmologe. 2020; 117(10):973-988. DOI: 10.1007/s00347-020-01209-z. View

10.

Omoumi P, Ducarouge A, Tournier A, Harvey H, Kahn Jr C, Louvet-de Verchere F . To buy or not to buy-evaluating commercial AI solutions in radiology (the ECLAIR guidelines). Eur Radiol. 2021; 31(6):3786-3796. PMC: 8128726. DOI: 10.1007/s00330-020-07684-x. View

11.

Taylor M, Liu X, Denniston A, Esteva A, Ko J, Daneshjou R . Raising the Bar for Randomized Trials Involving Artificial Intelligence: The SPIRIT-Artificial Intelligence and CONSORT-Artificial Intelligence Guidelines. J Invest Dermatol. 2021; 141(9):2109-2111. DOI: 10.1016/j.jid.2021.02.744. View

12.

Collins G, Moons K . Reporting of artificial intelligence prediction models. Lancet. 2019; 393(10181):1577-1579. DOI: 10.1016/S0140-6736(19)30037-6. View

13.

Collins G, Dhiman P, Andaur Navarro C, Ma J, Hooft L, Reitsma J . Protocol for development of a reporting guideline (TRIPOD-AI) and risk of bias tool (PROBAST-AI) for diagnostic and prognostic prediction model studies based on artificial intelligence. BMJ Open. 2021; 11(7):e048008. PMC: 8273461. DOI: 10.1136/bmjopen-2020-048008. View

14.

Sengupta P, Shrestha S, Berthon B, Messas E, Donal E, Tison G . Proposed Requirements for Cardiovascular Imaging-Related Machine Learning Evaluation (PRIME): A Checklist: Reviewed by the American College of Cardiology Healthcare Innovation Council. JACC Cardiovasc Imaging. 2020; 13(9):2017-2035. PMC: 7953597. DOI: 10.1016/j.jcmg.2020.07.015. View

15.

. DECIDE-AI: new reporting guidelines to bridge the development-to-implementation gap in clinical artificial intelligence. Nat Med. 2021; 27(2):186-187. DOI: 10.1038/s41591-021-01229-5. View

16.

Elmore J, Barnhill R, Elder D, Longton G, Pepe M, Reisch L . Pathologists' diagnosis of invasive melanoma and melanocytic proliferations: observer accuracy and reproducibility study. BMJ. 2017; 357:j2813. PMC: 5485913. DOI: 10.1136/bmj.j2813. View

17.

Tschandl P, Rinner C, Apalla Z, Argenziano G, Codella N, Halpern A . Human-computer collaboration for skin cancer recognition. Nat Med. 2020; 26(8):1229-1234. DOI: 10.1038/s41591-020-0942-0. View

18.

Petzold A, Albrecht P, Balcer L, Bekkers E, Brandt A, Calabresi P . Artificial intelligence extension of the OSCAR-IB criteria. Ann Clin Transl Neurol. 2021; 8(7):1528-1542. PMC: 8283174. DOI: 10.1002/acn3.51320. View

19.

Adamson A, Smith A . Machine Learning and Health Care Disparities in Dermatology. JAMA Dermatol. 2018; 154(11):1247-1248. DOI: 10.1001/jamadermatol.2018.2348. View

20.

Ibrahim H, Liu X, Cruz Rivera S, Moher D, Chan A, Sydes M . Reporting guidelines for clinical trials of artificial intelligence interventions: the SPIRIT-AI and CONSORT-AI guidelines. Trials. 2021; 22(1):11. PMC: 7788716. DOI: 10.1186/s13063-020-04951-6. View