A Visual Object Segmentation Algorithm with Spatial and Temporal Coherence Inspired by the Architecture of the Visual Cortex

Overview

Journal Cogn Process

Specialty Psychology

Date 2021 Nov 15

PMID 34779948

Authors

Juan A Ramirez-Quintana

Raul Rangel-Gonzalez

Mario I Chacon-Murguia

Graciela Ramirez-Alonso

Affiliations

Soon will be listed here.

Abstract

Scene analysis in video sequences is a complex task for a computer vision system. Several schemes have been addressed in this analysis, such as deep learning networks or traditional image processing methods. However, these methods require thorough training or manual adjustment of parameters to achieve accurate results. Therefore, it is necessary to develop novel methods to analyze the scenario information in video sequences. For this reason, this paper proposes a method for object segmentation in video sequences inspired by the structural layers of the visual cortex. The method is called Neuro-Inspired Object Segmentation, SegNI. SegNI has a hierarchical architecture that analyzes object features such as edges, color, and motion to generate regions that represent the objects in the scenario. The results obtained with the Video Segmentation Benchmark VSB100 dataset demonstrate that SegNI can adapt automatically to videos with scenarios that have different nature, composition, and different types of objects. Also, SegNI adapts its processing to new scenario conditions without training, which is a significant advantage over deep learning networks.

References

Arbelaez P, Maire M, Fowlkes C, Malik J . Contour detection and hierarchical image segmentation. IEEE Trans Pattern Anal Mach Intell. 2010; 33(5):898-916. DOI: 10.1109/TPAMI.2010.161. View

Bednar J, MIIKKULAINEN R . Tilt aftereffects in a self-organizing model of the primary visual cortex. Neural Comput. 2000; 12(7):1721-40. DOI: 10.1162/089976600300015321. View

Brito da Silva L, Elnabarawy I, Wunsch 2nd D . A survey of adaptive resonance theory neural network models for engineering applications. Neural Netw. 2019; 120:167-203. DOI: 10.1016/j.neunet.2019.09.012. View

Corso J, Sharon E, Dube S, El-Saden S, Sinha U, Yuille A . Efficient multilevel brain tumor segmentation with integrated bayesian model classification. IEEE Trans Med Imaging. 2008; 27(5):629-40. DOI: 10.1109/TMI.2007.912817. View

Kruger N, Janssen P, Kalkan S, Lappe M, Leonardis A, Piater J . Deep hierarchies in the primate visual cortex: what can we learn for computer vision?. IEEE Trans Pattern Anal Mach Intell. 2013; 35(8):1847-71. DOI: 10.1109/TPAMI.2012.272. View

Minaee S, Boykov Y, Porikli F, Plaza A, Kehtarnavaz N, Terzopoulos D . Image Segmentation Using Deep Learning: A Survey. IEEE Trans Pattern Anal Mach Intell. 2021; 44(7):3523-3542. DOI: 10.1109/TPAMI.2021.3059968. View

Sengupta N, McNabb C, Kasabov N, Russell B . Integrating Space, Time, and Orientation in Spiking Neural Networks: A Case Study on Multimodal Brain Data Modeling. IEEE Trans Neural Netw Learn Syst. 2018; 29(11):5249-5263. DOI: 10.1109/TNNLS.2018.2796023. View

Stoll S, Finlayson N, Schwarzkopf D . Topographic signatures of global object perception in human visual cortex. Neuroimage. 2020; 220:116926. PMC: 7573540. DOI: 10.1016/j.neuroimage.2020.116926. View

Thwaites A, Wingfield C, Wieser E, Soltan A, Marslen-Wilson W, Nimmo-Smith I . Entrainment to the CIECAM02 and CIELAB colour appearance models in the human cortex. Vision Res. 2018; 145:1-10. DOI: 10.1016/j.visres.2018.01.011. View

10.

Wang Q, Gao J, Li X . Weakly Supervised Adversarial Domain Adaptation for Semantic Segmentation in Urban Scenes. IEEE Trans Image Process. 2019; 28(9):4376-4386. DOI: 10.1109/TIP.2019.2910667. View

11.

Wang Z, Wang Z . A generic approach for cell segmentation based on Gabor filtering and area-constrained ultimate erosion. Artif Intell Med. 2020; 107:101929. DOI: 10.1016/j.artmed.2020.101929. View

12.

Yang K, Hu X, Stiefelhagen R . Is Context-Aware CNN Ready for the Surroundings? Panoramic Semantic Segmentation in the Wild. IEEE Trans Image Process. 2021; 30:1866-1881. DOI: 10.1109/TIP.2020.3048682. View

13.

Yu B, Zhang L . Pulse-coupled neural networks for contour and motion matchings. IEEE Trans Neural Netw. 2004; 15(5):1186-201. DOI: 10.1109/TNN.2004.832830. View