Understanding of Facial Features in Face Perception: Insights from Deep Convolutional Neural Networks

Overview

Journal Front Comput Neurosci

Specialty Biology

Date 2024 Apr 24

PMID 38655070

Authors

Qianqian Zhang

Yueyi Zhang

Ning Liu

Xiaoyan Sun

Affiliations

Soon will be listed here.

Abstract

Introduction: Face recognition has been a longstanding subject of interest in the fields of cognitive neuroscience and computer vision research. One key focus has been to understand the relative importance of different facial features in identifying individuals. Previous studies in humans have demonstrated the crucial role of eyebrows in face recognition, potentially even surpassing the importance of the eyes. However, eyebrows are not only vital for face recognition but also play a significant role in recognizing facial expressions and intentions, which might occur simultaneously and influence the face recognition process.

Methods: To address these challenges, our current study aimed to leverage the power of deep convolutional neural networks (DCNNs), an artificial face recognition system, which can be specifically tailored for face recognition tasks. In this study, we investigated the relative importance of various facial features in face recognition by selectively blocking feature information from the input to the DCNN. Additionally, we conducted experiments in which we systematically blurred the information related to eyebrows to varying degrees.

Results: Our findings aligned with previous human research, revealing that eyebrows are the most critical feature for face recognition, followed by eyes, mouth, and nose, in that order. The results demonstrated that the presence of eyebrows was more crucial than their specific high-frequency details, such as edges and textures, compared to other facial features, where the details also played a significant role. Furthermore, our results revealed that, unlike other facial features, the activation map indicated that the significance of eyebrows areas could not be readily adjusted to compensate for the absence of eyebrow information. This finding explains why masking eyebrows led to more significant deficits in face recognition performance. Additionally, we observed a synergistic relationship among facial features, providing evidence for holistic processing of faces within the DCNN.

Discussion: Overall, our study sheds light on the underlying mechanisms of face recognition and underscores the potential of using DCNNs as valuable tools for further exploration in this field.

References

Sadr J, Jarudi I, Sinha P . The role of eyebrows in face recognition. Perception. 2003; 32(3):285-93. DOI: 10.1068/p5027. View

Deng J, Guo J, Yang J, Xue N, Kotsia I, Zafeiriou S . ArcFace: Additive Angular Margin Loss for Deep Face Recognition. IEEE Trans Pattern Anal Mach Intell. 2021; 44(10):5962-5979. DOI: 10.1109/TPAMI.2021.3087709. View

Tanaka J, Simonyi D . The "parts and wholes" of face recognition: A review of the literature. Q J Exp Psychol (Hove). 2016; 69(10):1876-89. PMC: 5051945. DOI: 10.1080/17470218.2016.1146780. View

Yamins D, Hong H, Cadieu C, Solomon E, Seibert D, DiCarlo J . Performance-optimized hierarchical models predict neural responses in higher visual cortex. Proc Natl Acad Sci U S A. 2014; 111(23):8619-24. PMC: 4060707. DOI: 10.1073/pnas.1403112111. View

Bar M, Neta M, Linz H . Very first impressions. Emotion. 2006; 6(2):269-78. DOI: 10.1037/1528-3542.6.2.269. View

Richler J, Gauthier I . A meta-analysis and review of holistic face processing. Psychol Bull. 2014; 140(5):1281-302. PMC: 4152424. DOI: 10.1037/a0037004. View

Murphy J, Gray K, Cook R . The composite face illusion. Psychon Bull Rev. 2016; 24(2):245-261. DOI: 10.3758/s13423-016-1131-5. View

Grossman S, Gaziv G, Yeagle E, Harel M, Megevand P, Groppe D . Convergent evolution of face spaces across human face-selective neuronal groups and deep convolutional networks. Nat Commun. 2019; 10(1):4934. PMC: 6821842. DOI: 10.1038/s41467-019-12623-6. View

Antonakis J, Dalgas O . Predicting elections: child's play!. Science. 2009; 323(5918):1183. DOI: 10.1126/science.1167748. View

10.

Willis J, Todorov A . First impressions: making up your mind after a 100-ms exposure to a face. Psychol Sci. 2006; 17(7):592-8. DOI: 10.1111/j.1467-9280.2006.01750.x. View

11.

Tian F, Xie H, Song Y, Hu S, Liu J . The Face Inversion Effect in Deep Convolutional Neural Networks. Front Comput Neurosci. 2022; 16:854218. PMC: 9124772. DOI: 10.3389/fncom.2022.854218. View

12.

Chakravarthi B, Ng S, Ezilarasan M, Leung M . EEG-based emotion recognition using hybrid CNN and LSTM classification. Front Comput Neurosci. 2022; 16:1019776. PMC: 9585893. DOI: 10.3389/fncom.2022.1019776. View

13.

Jiang M, Francis S, Srishyla D, Conelea C, Zhao Q, Jacob S . Classifying Individuals with ASD Through Facial Emotion Recognition and Eye-Tracking. Annu Int Conf IEEE Eng Med Biol Soc. 2020; 2019:6063-6068. DOI: 10.1109/EMBC.2019.8857005. View

14.

Cadieu C, Hong H, Yamins D, Pinto N, Ardila D, Solomon E . Deep neural networks rival the representation of primate IT cortex for core visual object recognition. PLoS Comput Biol. 2014; 10(12):e1003963. PMC: 4270441. DOI: 10.1371/journal.pcbi.1003963. View

15.

Lim J, Mountstephens J, Teo J . Emotion Recognition Using Eye-Tracking: Taxonomy, Review and Current Challenges. Sensors (Basel). 2020; 20(8). PMC: 7219342. DOI: 10.3390/s20082384. View

16.

Haig N . Exploring recognition with interchanged facial features. Perception. 1986; 15(3):235-47. DOI: 10.1068/p150235. View

17.

Davies G, Ellis H, Shepherd J . Cue saliency in faces as assessed by the "Photofit" technique. Perception. 1977; 6(3):263-9. DOI: 10.1068/p060263. View

18.

Richler J, Palmeri T, Gauthier I . Meanings, mechanisms, and measures of holistic processing. Front Psychol. 2012; 3:553. PMC: 3520179. DOI: 10.3389/fpsyg.2012.00553. View

19.

Jacob G, Pramod R, Katti H, Arun S . Qualitative similarities and differences in visual object representations between brains and deep networks. Nat Commun. 2021; 12(1):1872. PMC: 7994307. DOI: 10.1038/s41467-021-22078-3. View

20.

Young A, Hellawell D, Hay D . Configurational information in face perception. Perception. 1987; 16(6):747-59. DOI: 10.1068/p160747. View