The Multimodality Cell Segmentation Challenge: Toward Universal Solutions

Overview

Journal Nat Methods

Specialties Biomedical Engineering
Pathology

Date 2024 Mar 27

PMID 38532015

Authors

Jun Ma

Ronald Xie

Shamini Ayyadhury

Cheng Ge

Anubha Gupta

Ritu Gupta

Song Gu

Yao Zhang

Gihun Lee

Joonkee Kim

Wei Lou

Haofeng Li

Eric Upschulte

Timo Dickscheid

Jose Guilherme de Almeida

Yixin Wang

Lin Han

Xin Yang

Marco Labagnara

Vojislav Gligorovski

Maxime Scheder

Sahand Jamal Rahi

Carly Kempster

Alice Pollitt

Leon Espinosa

Tam Mignot

Jan Moritz Middeke

Jan-Niklas Eckardt

Wangkai Li

Zhaoyang Li

Xiaochen Cai

Bizhe Bai

Noah F Greenwald

David Van Valen

Erin Weisbart

Beth A Cimini

Trevor Cheung

Oscar Bruck

Gary D Bader

Bo Wang

Affiliations

Soon will be listed here.

Abstract

Cell segmentation is a critical step for quantitative single-cell analysis in microscopy images. Existing cell segmentation methods are often tailored to specific modalities or require manual interventions to specify hyper-parameters in different experimental settings. Here, we present a multimodality cell segmentation benchmark, comprising more than 1,500 labeled images derived from more than 50 diverse biological experiments. The top participants developed a Transformer-based deep-learning algorithm that not only exceeds existing methods but can also be applied to diverse microscopy images across imaging platforms and tissue types without manual parameter adjustments. This benchmark and the improved algorithm offer promising avenues for more accurate and versatile cell analysis in microscopy imaging.

Citing Articles

Automated deep learning-based assessment of tumour-infiltrating lymphocyte density determines prognosis in colorectal cancer.

Millward J, He Z, Nibali A, Mouradov D, Mielke L, Tran K J Transl Med. 2025; 23(1):298.

PMID: 40065354 PMC: 11892243. DOI: 10.1186/s12967-025-06254-3.

Toward spatial glycomics and glycoproteomics: Innovations and applications.

Boottanun P, Fuseya S, Kuno A BBA Adv. 2025; 7:100146.

PMID: 40027887 PMC: 11869499. DOI: 10.1016/j.bbadva.2025.100146.

Advancing antibiotic discovery with bacterial cytological profiling: a high-throughput solution to antimicrobial resistance.

Salgado J, Rayner J, Ojkic N Front Microbiol. 2025; 16:1536131.

PMID: 40018674 PMC: 11865948. DOI: 10.3389/fmicb.2025.1536131.

Cellpose3: one-click image restoration for improved cellular segmentation.

Stringer C, Pachitariu M Nat Methods. 2025; 22(3):592-599.

PMID: 39939718 PMC: 11903308. DOI: 10.1038/s41592-025-02595-5.

Segment Anything for Microscopy.

Archit A, Freckmann L, Nair S, Khalid N, Hilt P, Rajashekar V Nat Methods. 2025; 22(3):579-591.

PMID: 39939717 PMC: 11903314. DOI: 10.1038/s41592-024-02580-4.

References

Cutler K, Stringer C, Lo T, Rappez L, Stroustrup N, Peterson S . Omnipose: a high-precision morphology-independent solution for bacterial cell segmentation. Nat Methods. 2022; 19(11):1438-1448. PMC: 9636021. DOI: 10.1038/s41592-022-01639-4. View

Ulman V, Maska M, Magnusson K, Ronneberger O, Haubold C, Harder N . An objective comparison of cell-tracking algorithms. Nat Methods. 2017; 14(12):1141-1152. PMC: 5777536. DOI: 10.1038/nmeth.4473. View

Upschulte E, Harmeling S, Amunts K, Dickscheid T . Contour proposal networks for biomedical instance segmentation. Med Image Anal. 2022; 77:102371. DOI: 10.1016/j.media.2022.102371. View

Risom T, Glass D, Averbukh I, Liu C, Baranski A, Kagel A . Transition to invasive breast cancer is associated with progressive changes in the structure and composition of tumor stroma. Cell. 2022; 185(2):299-310.e18. PMC: 8792442. DOI: 10.1016/j.cell.2021.12.023. View

Falk T, Mai D, Bensch R, Cicek O, Abdulkadir A, Marrakchi Y . U-Net: deep learning for cell counting, detection, and morphometry. Nat Methods. 2018; 16(1):67-70. DOI: 10.1038/s41592-018-0261-2. View

Chen L, Papandreou G, Kokkinos I, Murphy K, Yuille A . DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs. IEEE Trans Pattern Anal Mach Intell. 2017; 40(4):834-848. DOI: 10.1109/TPAMI.2017.2699184. View

Maska M, Ulman V, Delgado-Rodriguez P, Gomez-de-Mariscal E, Necasova T, Guerrero Pena F . The Cell Tracking Challenge: 10 years of objective benchmarking. Nat Methods. 2023; 20(7):1010-1020. PMC: 10333123. DOI: 10.1038/s41592-023-01879-y. View

Ma J, He Y, Li F, Han L, You C, Wang B . Segment anything in medical images. Nat Commun. 2024; 15(1):654. PMC: 10803759. DOI: 10.1038/s41467-024-44824-z. View

Jackson H, Fischer J, Zanotelli V, Ali H, Mechera R, Soysal S . The single-cell pathology landscape of breast cancer. Nature. 2020; 578(7796):615-620. DOI: 10.1038/s41586-019-1876-x. View

10.

Maier-Hein L, Eisenmann M, Reinke A, Onogur S, Stankovic M, Scholz P . Why rankings of biomedical image analysis competitions should be interpreted with care. Nat Commun. 2018; 9(1):5217. PMC: 6284017. DOI: 10.1038/s41467-018-07619-7. View

11.

Ren S, He K, Girshick R, Sun J . Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. IEEE Trans Pattern Anal Mach Intell. 2016; 39(6):1137-1149. DOI: 10.1109/TPAMI.2016.2577031. View

12.

Bunk D, Moriasy J, Thoma F, Jakubke C, Osman C, Horl D . YeastMate: neural network-assisted segmentation of mating and budding events in Saccharomyces cerevisiae. Bioinformatics. 2022; 38(9):2667-2669. PMC: 9048668. DOI: 10.1093/bioinformatics/btac107. View

13.

Lin J, Wang S, Coy S, Chen Y, Yapp C, Tyler M . Multiplexed 3D atlas of state transitions and immune interaction in colorectal cancer. Cell. 2023; 186(2):363-381.e19. PMC: 10019067. DOI: 10.1016/j.cell.2022.12.028. View

14.

Fu S, Shi W, Luo T, He Y, Zhou L, Yang J . Field-dependent deep learning enables high-throughput whole-cell 3D super-resolution imaging. Nat Methods. 2023; 20(3):459-468. DOI: 10.1038/s41592-023-01775-5. View

15.

Ma J, Wang B . Towards foundation models of biological image segmentation. Nat Methods. 2023; 20(7):953-955. DOI: 10.1038/s41592-023-01885-0. View

16.

Caicedo J, Goodman A, Karhohs K, Cimini B, Ackerman J, Haghighi M . Nucleus segmentation across imaging experiments: the 2018 Data Science Bowl. Nat Methods. 2019; 16(12):1247-1253. PMC: 6919559. DOI: 10.1038/s41592-019-0612-7. View

17.

Hirling D, Tasnadi E, Caicedo J, Caroprese M, Sjogren R, Aubreville M . Segmentation metric misinterpretations in bioimage analysis. Nat Methods. 2023; 21(2):213-216. PMC: 10864175. DOI: 10.1038/s41592-023-01942-8. View

18.

Parisi G, Kemker R, Part J, Kanan C, Wermter S . Continual lifelong learning with neural networks: A review. Neural Netw. 2019; 113:54-71. DOI: 10.1016/j.neunet.2019.01.012. View

19.

Gupta A, Gehlot S, Goswami S, Motwani S, Gupta R, Faura A . SegPC-2021: A challenge & dataset on segmentation of Multiple Myeloma plasma cells from microscopic images. Med Image Anal. 2022; 83:102677. DOI: 10.1016/j.media.2022.102677. View

20.

Scherr T, Loffler K, Bohland M, Mikut R . Cell segmentation and tracking using CNN-based distance predictions and a graph-based matching strategy. PLoS One. 2020; 15(12):e0243219. PMC: 7723299. DOI: 10.1371/journal.pone.0243219. View