The Liver Tumor Segmentation Benchmark (LiTS)

In this work, we report the set-up and results of the Liver Tumor Segmentation Benchmark (LiTS), which was organized in conjunction with the IEEE International Symposium on Biomedical Imaging (ISBI) 2017 and the International Conferences on Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2017 and 2018. The image dataset is diverse and contains primary and secondary tumors with varied sizes and appearances with various lesion-to-background levels (hyper-/hypo-dense), created in collaboration with seven hospitals and research institutions. Seventy-five submitted liver and liver tumor segmentation algorithms were trained on a set of 131 computed tomography (CT) volumes and were tested on 70 unseen test images acquired from different patients. We found that not a single algorithm performed best for both liver and liver tumors in the three events. The best liver segmentation algorithm achieved a Dice score of 0.963, whereas, for tumor segmentation, the best algorithms achieved Dices scores of 0.674 (ISBI 2017), 0.702 (MICCAI 2017), and 0.739 (MICCAI 2018). Retrospectively, we performed additional analysis on liver tumor detection and revealed that not all top-performing segmentation algorithms worked well for tumor detection. The best liver tumor detection method achieved a lesion-wise recall of 0.458 (ISBI 2017), 0.515 (MICCAI 2017), and 0.554 (MICCAI 2018), indicating the need for further research. LiTS remains an active benchmark and resource for research, e.g., contributing the liver-related segmentation tasks in http://medicaldecathlon.com/. In addition, both data and online evaluation are accessible via https://competitions.codalab.org/competitions/17094.

Citing Articles

Rethinking model prototyping through the MedMNIST+ dataset collection.

Doerrich S, Di Salvo F, Brockmann J, Ledig C Sci Rep. 2025; 15(1):7669.

PMID: 40044786 PMC: 11883007. DOI: 10.1038/s41598-025-92156-9.

Systematic Review: AI Applications in Liver Imaging with a Focus on Segmentation and Detection.

Pomohaci M, Grasu M, Baicoianu-Nitescu A, Enache R, Lupescu I Life (Basel). 2025; 15(2).

PMID: 40003667 PMC: 11856300. DOI: 10.3390/life15020258.

Anatomically Guided Deep Learning System for Right Internal Jugular Line (RIJL) Segmentation and Tip Localization in Chest X-Ray.

Wei S, Shrestha L, Melendez-Corres G, Brown M Life (Basel). 2025; 15(2).

PMID: 40003610 PMC: 11856827. DOI: 10.3390/life15020201.

Automatic future remnant segmentation in liver resection planning.

Messaoudi H, Abbas M, Badic B, Ben Salem D, Belaid A, Conze P Int J Comput Assist Radiol Surg. 2025; .

PMID: 39961898 DOI: 10.1007/s11548-025-03331-2.

Enhancing unsupervised learning in medical image registration through scale-aware context aggregation.

Liu Y, Wang L, Ning X, Gao Y, Wang D iScience. 2025; 28(2):111734.

PMID: 39898031 PMC: 11787544. DOI: 10.1016/j.isci.2024.111734.

References

Shiina S, Sato K, Tateishi R, Shimizu M, Ohama H, Hatanaka T . Percutaneous Ablation for Hepatocellular Carcinoma: Comparison of Various Ablation Techniques and Surgery. Can J Gastroenterol Hepatol. 2018; 2018:4756147. PMC: 6008833. DOI: 10.1155/2018/4756147. View

Xu Z, Burke R, Lee C, Baucom R, Poulose B, Abramson R . Efficient multi-atlas abdominal segmentation on clinically acquired CT with SIMPLE context learning. Med Image Anal. 2015; 24(1):18-27. PMC: 4532551. DOI: 10.1016/j.media.2015.05.009. View

Linguraru M, Richbourg W, Liu J, Watt J, Pamulapati V, Wang S . Tumor burden analysis on computed tomography by automated liver and tumor segmentation. IEEE Trans Med Imaging. 2012; 31(10):1965-76. PMC: 3924860. DOI: 10.1109/TMI.2012.2211887. View

Albain K, Swann R, Rusch V, Turrisi 3rd A, Shepherd F, Smith C . Radiotherapy plus chemotherapy with or without surgical resection for stage III non-small-cell lung cancer: a phase III randomised controlled trial. Lancet. 2009; 374(9687):379-86. PMC: 4407808. DOI: 10.1016/S0140-6736(09)60737-6. View

Li H, Jiang G, Zhang J, Wang R, Wang Z, Zheng W . Fully convolutional network ensembles for white matter hyperintensities segmentation in MR images. Neuroimage. 2018; 183:650-665. DOI: 10.1016/j.neuroimage.2018.07.005. View

Cano-Espinosa C, Gonzalez G, Washko G, Cazorla M, San Jose Estepar R . Biomarker Localization From Deep Learning Regression Networks. IEEE Trans Med Imaging. 2020; 39(6):2121-2132. PMC: 7307703. DOI: 10.1109/TMI.2020.2965486. View

Fang X, Yan P . Multi-Organ Segmentation Over Partially Labeled Datasets With Multi-Scale Feature Abstraction. IEEE Trans Med Imaging. 2020; 39(11):3619-3629. PMC: 7665851. DOI: 10.1109/TMI.2020.3001036. View

Castro D, Walker I, Glocker B . Causality matters in medical imaging. Nat Commun. 2020; 11(1):3673. PMC: 7376027. DOI: 10.1038/s41467-020-17478-w. View

Litjens G, Kooi T, Bejnordi B, Setio A, Ciompi F, Ghafoorian M . A survey on deep learning in medical image analysis. Med Image Anal. 2017; 42:60-88. DOI: 10.1016/j.media.2017.07.005. View

10.

Heimann T, Wolf I, Meinzer H . Active shape models for a fully automated 3D segmentation of the liver--an evaluation on clinical data. Med Image Comput Comput Assist Interv. 2007; 9(Pt 2):41-8. DOI: 10.1007/11866763_6. View

11.

Schoppe O, Pan C, Coronel J, Mai H, Rong Z, Todorov M . Deep learning-enabled multi-organ segmentation in whole-body mouse scans. Nat Commun. 2020; 11(1):5626. PMC: 7648799. DOI: 10.1038/s41467-020-19449-7. View

12.

Eisenhauer E, Therasse P, Bogaerts J, Schwartz L, Sargent D, Ford R . New response evaluation criteria in solid tumours: revised RECIST guideline (version 1.1). Eur J Cancer. 2008; 45(2):228-47. DOI: 10.1016/j.ejca.2008.10.026. View

13.

Park H, Bland P, Meyer C . Construction of an abdominal probabilistic atlas and its application in segmentation. IEEE Trans Med Imaging. 2003; 22(4):483-92. DOI: 10.1109/TMI.2003.809139. View

14.

Heimann T, van Ginneken B, Styner M, Arzhaeva Y, Aurich V, Bauer C . Comparison and evaluation of methods for liver segmentation from CT datasets. IEEE Trans Med Imaging. 2009; 28(8):1251-65. DOI: 10.1109/TMI.2009.2013851. View

15.

Todorov M, Paetzold J, Schoppe O, Tetteh G, Shit S, Efremov V . Machine learning analysis of whole mouse brain vasculature. Nat Methods. 2020; 17(4):442-449. PMC: 7591801. DOI: 10.1038/s41592-020-0792-1. View

16.

Haghighi F, Taher M, Zhou Z, Gotway M, Liang J . Transferable Visual Words: Exploiting the Semantics of Anatomical Patterns for Self-Supervised Learning. IEEE Trans Med Imaging. 2021; 40(10):2857-2868. PMC: 8516596. DOI: 10.1109/TMI.2021.3060634. View

17.

Ma J, He J, Yang X . Learning Geodesic Active Contours for Embedding Object Global Information in Segmentation CNNs. IEEE Trans Med Imaging. 2020; 40(1):93-104. DOI: 10.1109/TMI.2020.3022693. View

18.

Puesken M, Buerke B, Gerss J, Frisch B, Beyer F, Weckesser M . Prediction of lymph node manifestations in malignant lymphoma: significant role of volumetric compared with established metric lymph node analysis in multislice computed tomography. J Comput Assist Tomogr. 2010; 34(4):564-9. DOI: 10.1097/RCT.0b013e3181db2901. View

19.

Chlebus G, Schenk A, Moltz J, van Ginneken B, Hahn H, Meine H . Automatic liver tumor segmentation in CT with fully convolutional neural networks and object-based postprocessing. Sci Rep. 2018; 8(1):15497. PMC: 6195599. DOI: 10.1038/s41598-018-33860-7. View

20.

Kuhnigk J, Dicken V, Bornemann L, Bakai A, Wormanns D, Krass S . Morphological segmentation and partial volume analysis for volumetry of solid pulmonary lesions in thoracic CT scans. IEEE Trans Med Imaging. 2006; 25(4):417-34. DOI: 10.1109/TMI.2006.871547. View