» Articles » PMID: 37665496

A Novel Deep Learning Model for Medical Image Segmentation with Convolutional Neural Network and Transformer

Overview
Journal Interdiscip Sci
Specialty Biology
Date 2023 Sep 4
PMID 37665496
Authors
Affiliations
Soon will be listed here.
Abstract

Accurate segmentation of medical images is essential for clinical decision-making, and deep learning techniques have shown remarkable results in this area. However, existing segmentation models that combine transformer and convolutional neural networks often use skip connections in U-shaped networks, which may limit their ability to capture contextual information in medical images. To address this limitation, we propose a coordinated mobile and residual transformer UNet (MRC-TransUNet) that combines the strengths of transformer and UNet architectures. Our approach uses a lightweight MR-ViT to address the semantic gap and a reciprocal attention module to compensate for the potential loss of details. To better explore long-range contextual information, we use skip connections only in the first layer and add MR-ViT and RPA modules in the subsequent downsampling layers. In our study, we evaluated the effectiveness of our proposed method on three different medical image segmentation datasets, namely, breast, brain, and lung. Our proposed method outperformed state-of-the-art methods in terms of various evaluation metrics, including the Dice coefficient and Hausdorff distance. These results demonstrate that our proposed method can significantly improve the accuracy of medical image segmentation and has the potential for clinical applications. Illustration of the proposed MRC-TransUNet. For the input medical images, we first subject them to an intrinsic downsampling operation and then replace the original jump connection structure using MR-ViT. The output feature representations at different scales are fused by the RPA module. Finally, an upsampling operation is performed to fuse the features to restore them to the same resolution as the input image.

Citing Articles

Detection of Masses in Mammogram Images Based on the Enhanced RetinaNet Network With INbreast Dataset.

Wang M, Liu R, Luttrell Iv J, Zhang C, Xie J J Multidiscip Healthc. 2025; 18:675-695.

PMID: 39935433 PMC: 11812562. DOI: 10.2147/JMDH.S493873.


Deep learning-based automated tool for diagnosing diabetic peripheral neuropathy.

Qiao Q, Cao J, Xue W, Qian J, Wang C, Pan Q Digit Health. 2025; 10:20552076241307573.

PMID: 39741986 PMC: 11686633. DOI: 10.1177/20552076241307573.


MARes-Net: multi-scale attention residual network for jaw cyst image segmentation.

Ding X, Jiang X, Zheng H, Shi H, Wang B, Chan S Front Bioeng Biotechnol. 2024; 12:1454728.

PMID: 39161348 PMC: 11330813. DOI: 10.3389/fbioe.2024.1454728.


Advances in research and application of artificial intelligence and radiomic predictive models based on intracranial aneurysm images.

Wen Z, Wang Y, Zhong Y, Hu Y, Yang C, Peng Y Front Neurol. 2024; 15:1391382.

PMID: 38694771 PMC: 11061371. DOI: 10.3389/fneur.2024.1391382.


An ensemble model for predicting dispositions of emergency department patients.

Kuo K, Lin Y, Chang C, Kuo T BMC Med Inform Decis Mak. 2024; 24(1):105.

PMID: 38649949 PMC: 11036695. DOI: 10.1186/s12911-024-02503-5.

References
1.
Prastawa M, Bullitt E, Gerig G . Simulation of brain tumors in MR images for evaluation of segmentation efficacy. Med Image Anal. 2009; 13(2):297-311. PMC: 2660387. DOI: 10.1016/j.media.2008.11.002. View

2.
Corso J, Sharon E, Dube S, El-Saden S, Sinha U, Yuille A . Efficient multilevel brain tumor segmentation with integrated bayesian model classification. IEEE Trans Med Imaging. 2008; 27(5):629-40. DOI: 10.1109/TMI.2007.912817. View

3.
Guerrero R, Qin C, Oktay O, Bowles C, Chen L, Joules R . White matter hyperintensity and stroke lesion segmentation and differentiation using convolutional neural networks. Neuroimage Clin. 2018; 17:918-934. PMC: 5842732. DOI: 10.1016/j.nicl.2017.12.022. View

4.
Zhang Z, Zhang X, Yang Y, Liu J, Zheng C, Bai H . Accurate segmentation algorithm of acoustic neuroma in the cerebellopontine angle based on ACP-TransUNet. Front Neurosci. 2023; 17:1207149. PMC: 10244508. DOI: 10.3389/fnins.2023.1207149. View

5.
Ibtehaz N, Sohel Rahman M . MultiResUNet : Rethinking the U-Net architecture for multimodal biomedical image segmentation. Neural Netw. 2019; 121:74-87. DOI: 10.1016/j.neunet.2019.08.025. View