Spine-GAN: Semantic Segmentation of Multiple Spinal Structures

Spinal clinicians still rely on laborious workloads to conduct comprehensive assessments of multiple spinal structures in MRIs, in order to detect abnormalities and discover possible pathological factors. The objective of this work is to perform automated segmentation and classification (i.e., normal and abnormal) of intervertebral discs, vertebrae, and neural foramen in MRIs in one shot, which is called semantic segmentation that is extremely urgent to assist spinal clinicians in diagnosing neural foraminal stenosis, disc degeneration, and vertebral deformity as well as discovering possible pathological factors. However, no work has simultaneously achieved the semantic segmentation of intervertebral discs, vertebrae, and neural foramen due to three-fold unusual challenges: 1) Multiple tasks, i.e., simultaneous semantic segmentation of multiple spinal structures, are more difficult than individual tasks; 2) Multiple targets: average 21 spinal structures per MRI require automated analysis yet have high variety and variability; 3) Weak spatial correlations and subtle differences between normal and abnormal structures generate dynamic complexity and indeterminacy. In this paper, we propose a Recurrent Generative Adversarial Network called Spine-GAN for resolving above-aforementioned challenges. Firstly, Spine-GAN explicitly solves the high variety and variability of complex spinal structures through an atrous convolution (i.e., convolution with holes) autoencoder module that is capable of obtaining semantic task-aware representation and preserving fine-grained structural information. Secondly, Spine-GAN dynamically models the spatial pathological correlations between both normal and abnormal structures thanks to a specially designed long short-term memory module. Thirdly, Spine-GAN obtains reliable performance and efficient generalization by leveraging a discriminative network that is capable of correcting predicted errors and global-level contiguity. Extensive experiments on MRIs of 253 patients have demonstrated that Spine-GAN achieves high pixel accuracy of 96.2%, Dice coefficient of 87.1%, Sensitivity of 89.1% and Specificity of 86.0%, which reveals its effectiveness and potential as a clinical tool.