Static and Dynamic Hand Gestures: A Review of Techniques of Virtual Reality Manipulation

Overview

Journal Sensors (Basel)

Publisher MDPI

Specialty Biotechnology

Date 2024 Jun 27

PMID 38931542

Authors

Oswaldo Mendoza Herbert

David Perez-Granados

Mauricio Alberto Ortega Ruiz

Rodrigo Cadena Martinez

Carlos Alberto Gonzalez Gutierrez

Marco Antonio Zamora Antunano

Affiliations

Soon will be listed here.

Abstract

This review explores the historical and current significance of gestures as a universal form of communication with a focus on hand gestures in virtual reality applications. It highlights the evolution of gesture detection systems from the 1990s, which used computer algorithms to find patterns in static images, to the present day where advances in sensor technology, artificial intelligence, and computing power have enabled real-time gesture recognition. The paper emphasizes the role of hand gestures in virtual reality (VR), a field that creates immersive digital experiences through the Ma blending of 3D modeling, sound effects, and sensing technology. This review presents state-of-the-art hardware and software techniques used in hand gesture detection, primarily for VR applications. It discusses the challenges in hand gesture detection, classifies gestures as static and dynamic, and grades their detection difficulty. This paper also reviews the haptic devices used in VR and their advantages and challenges. It provides an overview of the process used in hand gesture acquisition, from inputs and pre-processing to pose detection, for both static and dynamic gestures.

References

Shimada K, Ikeda R, Kikura H, Takahashi H . Morphological Fabrication of Rubber Cutaneous Receptors Embedded in a Stretchable Skin-Mimicking Human Tissue by the Utilization of Hybrid Fluid. Sensors (Basel). 2021; 21(20). PMC: 8540356. DOI: 10.3390/s21206834. View

Sheng Y, Cheng H, Wang Y, Zhao H, Ding H . Teleoperated Surgical Robot with Adaptive Interactive Control Architecture for Tissue Identification. Bioengineering (Basel). 2023; 10(10). PMC: 10603885. DOI: 10.3390/bioengineering10101157. View

Wang P, Wu P, Wang J, Chi H, Wang X . A Critical Review of the Use of Virtual Reality in Construction Engineering Education and Training. Int J Environ Res Public Health. 2018; 15(6). PMC: 6025066. DOI: 10.3390/ijerph15061204. View

Parsons T, Gaggioli A, Riva G . Virtual Reality for Research in Social Neuroscience. Brain Sci. 2017; 7(4). PMC: 5406699. DOI: 10.3390/brainsci7040042. View

Chen Z, Huang W, Liu H, Wang Z, Wen Y, Wang S . ST-TGR: Spatio-Temporal Representation Learning for Skeleton-Based Teaching Gesture Recognition. Sensors (Basel). 2024; 24(8). PMC: 11054209. DOI: 10.3390/s24082589. View

Stenger B, Thayananthan A, Torr P, Cipolla R . Model-based hand tracking using a hierarchical Bayesian filter. IEEE Trans Pattern Anal Mach Intell. 2006; 28(9):1372-84. DOI: 10.1109/TPAMI.2006.189. View

Kuo L, Chiu H, Chang C, Hsu H, Sun Y . Functional workspace for precision manipulation between thumb and fingers in normal hands. J Electromyogr Kinesiol. 2008; 19(5):829-39. DOI: 10.1016/j.jelekin.2008.07.008. View

Squeri V, Sciutti A, Gori M, Masia L, Sandini G, Konczak J . Two hands, one perception: how bimanual haptic information is combined by the brain. J Neurophysiol. 2011; 107(2):544-50. DOI: 10.1152/jn.00756.2010. View

Han J, Shao L, Xu D, Shotton J . Enhanced computer vision with Microsoft Kinect sensor: a review. IEEE Trans Cybern. 2013; 43(5):1318-34. DOI: 10.1109/TCYB.2013.2265378. View

10.

Caeiro-Rodriguez M, Otero-Gonzalez I, Mikic-Fonte F, Llamas-Nistal M . A Systematic Review of Commercial Smart Gloves: Current Status and Applications. Sensors (Basel). 2021; 21(8). PMC: 8070066. DOI: 10.3390/s21082667. View

11.

Pacchierotti C, Sinclair S, Solazzi M, Frisoli A, Hayward V, Prattichizzo D . Wearable Haptic Systems for the Fingertip and the Hand: Taxonomy, Review, and Perspectives. IEEE Trans Haptics. 2017; 10(4):580-600. DOI: 10.1109/TOH.2017.2689006. View

12.

Yang K, Kim M, Jung Y, Lee S . Hand Gesture Recognition Using FSK Radar Sensors. Sensors (Basel). 2024; 24(2). PMC: 10820019. DOI: 10.3390/s24020349. View

13.

Sorgini F, Calio R, Carrozza M, Oddo C . Haptic-assistive technologies for audition and vision sensory disabilities. Disabil Rehabil Assist Technol. 2017; 13(4):394-421. DOI: 10.1080/17483107.2017.1385100. View

14.

Rose F, Brooks B, Rizzo A . Virtual reality in brain damage rehabilitation: review. Cyberpsychol Behav. 2005; 8(3):241-62. DOI: 10.1089/cpb.2005.8.241. View

15.

Maisto M, Pacchierotti C, Chinello F, Salvietti G, De Luca A, Prattichizzo D . Evaluation of Wearable Haptic Systems for the Fingers in Augmented Reality Applications. IEEE Trans Haptics. 2017; 10(4):511-522. DOI: 10.1109/TOH.2017.2691328. View

16.

Santello M, Baud-Bovy G, Jorntell H . Neural bases of hand synergies. Front Comput Neurosci. 2013; 7:23. PMC: 3619124. DOI: 10.3389/fncom.2013.00023. View

17.

Achenbach P, Laux S, Purdack D, Muller P, Gobel S . Give Me a Sign: Using Data Gloves for Static Hand-Shape Recognition. Sensors (Basel). 2023; 23(24). PMC: 10747392. DOI: 10.3390/s23249847. View

18.

Shull P, Damian D . Haptic wearables as sensory replacement, sensory augmentation and trainer - a review. J Neuroeng Rehabil. 2015; 12:59. PMC: 4506766. DOI: 10.1186/s12984-015-0055-z. View

19.

Ullrich S, Kuhlen T . Haptic palpation for medical simulation in virtual environments. IEEE Trans Vis Comput Graph. 2012; 18(4):617-25. DOI: 10.1109/TVCG.2012.46. View

20.

Abdallah M, Samaan G, Wadie A, Makhmudov F, Cho Y . Light-Weight Deep Learning Techniques with Advanced Processing for Real-Time Hand Gesture Recognition. Sensors (Basel). 2023; 23(1). PMC: 9823561. DOI: 10.3390/s23010002. View