Utilizing Eye Tracking to Assess Medical Student Non-Technical Performance During Scenario-Based Simulation: Results of a Pilot Study

Overview

Journal Global Surg Educ

Publisher Springer

Specialty Medical Education

Date 2024 Feb 28

PMID 38414559

Authors

Nicholas E Anton

Jackie S Cha

Edward Hernandez

Dimitrios Athanasiadis

Jing Yang

Guoyang Zhou

Dimitrios Stefanidis

Denny Yu

Affiliations

Soon will be listed here.

Abstract

Background: Non-technical skills (NTS) are essential for safe surgical patient management. However, assessing NTS involves observer-based ratings, which can introduce bias. Eye tracking (ET) has been proposed as an effective method to capture NTS. The purpose of the current study was to determine if ET metrics are associated with NTS performance.

Methods: Participants wore a mobile ET system and participated in two patient care simulations, where they managed a deteriorating patient. The scenarios featured several challenges to leadership, which were evaluated using a 4-point Likert scale. NTS were evaluated by trained raters using the Non-Technical Skills for Surgeons (NOTSS) scale. ET metrics included percentage of fixations and visits on areas of interest.

Results: Ten medical students participated. Average visit duration on the patient was negatively correlated with participants' communication and leadership. Average visit duration on the patient's intravenous access was negatively correlated with participants' decision making and situation awareness.

Conclusions: Our preliminary data suggests that visual attention on the patient was negatively associated with NTS and may indicate poor comprehension of the patient's status due to heightened cognitive load. In future work, researchers and educators should consider using ET to objectively evaluate and provide feedback on their NTS.

Citing Articles

Nontechnical Skills Assessment in Acute Care Trauma Simulations: A Mixed Methods Approach Using Eye Tracking and Behavioral Marker Systems.

Anton N, Obuseh M, Lim C, Chen H, Yang J, Stefanidis D Mil Med. 2024; 189(Suppl 3):719-727.

PMID: 39160814 PMC: 11368209. DOI: 10.1093/milmed/usae240.

Eye tracking and audio sensors to evaluate surgeon's non-technical skills: An empirical study.

Narasimha S, Obuseh M, Anton N, Chen H, Chakrabarty R, Stefanidis D Appl Ergon. 2024; 119:104320.

PMID: 38797012 PMC: 11863474. DOI: 10.1016/j.apergo.2024.104320.

References

Cha J, Athanasiadis D, Peng Y, Wu D, Anton N, Stefanidis D . Objective Nontechnical Skills Measurement Using Sensor-based Behavior Metrics in Surgical Teams. Hum Factors. 2022; 66(3):729-743. DOI: 10.1177/00187208221101292. View

Vioque S, Kim P, McMaster J, Gallagher J, Allen S, Holena D . Classifying errors in preventable and potentially preventable trauma deaths: a 9-year review using the Joint Commission's standardized methodology. Am J Surg. 2014; 208(2):187-94. DOI: 10.1016/j.amjsurg.2014.02.006. View

Capogna E, Capogna G, Raccis D, Salvi F, Velardo M, Del Vecchio A . Eye tracking metrics and leader's behavioral performance during a post-partum hemorrhage high-fidelity simulated scenario. Adv Simul (Lond). 2021; 6(1):4. PMC: 7863418. DOI: 10.1186/s41077-021-00156-2. View

Wu C, Cha J, Sulek J, Sundaram C, Wachs J, Proctor R . Sensor-based indicators of performance changes between sessions during robotic surgery training. Appl Ergon. 2020; 90:103251. PMC: 7606790. DOI: 10.1016/j.apergo.2020.103251. View

Yu D, Blocker R, Sir M, Hallbeck M, Hellmich T, Cohen T . Intelligent Emergency Department: Validation of Sociometers to Study Workload. J Med Syst. 2015; 40(3):53. DOI: 10.1007/s10916-015-0405-1. View

Hull L, Arora S, Aggarwal R, Darzi A, Vincent C, Sevdalis N . The impact of nontechnical skills on technical performance in surgery: a systematic review. J Am Coll Surg. 2011; 214(2):214-30. DOI: 10.1016/j.jamcollsurg.2011.10.016. View

Chetwood A, Kwok K, Sun L, Mylonas G, Clark J, Darzi A . Collaborative eye tracking: a potential training tool in laparoscopic surgery. Surg Endosc. 2012; 26(7):2003-9. DOI: 10.1007/s00464-011-2143-x. View

Yule S, Flin R, Maran N, Rowley D, Youngson G, Paterson-Brown S . Surgeons' non-technical skills in the operating room: reliability testing of the NOTSS behavior rating system. World J Surg. 2008; 32(4):548-56. DOI: 10.1007/s00268-007-9320-z. View

Catchpole K, Mishra A, Handa A, McCulloch P . Teamwork and error in the operating room: analysis of skills and roles. Ann Surg. 2008; 247(4):699-706. DOI: 10.1097/SLA.0b013e3181642ec8. View

10.

Wise R, Faurie M, Malbrain M, Hodgson E . Strategies for Intravenous Fluid Resuscitation in Trauma Patients. World J Surg. 2017; 41(5):1170-1183. PMC: 5394148. DOI: 10.1007/s00268-016-3865-7. View

11.

Vandemoortele S, Feyaerts K, Reybrouck M, De Bievre G, Brone G, De Baets T . Gazing at the partner in musical trios: a mobile eye-tracking study. J Eye Mov Res. 2021; 11(2). PMC: 7745343. DOI: 10.16910/jemr.11.2.6. View

12.

Gawande A, Zinner M, Studdert D, Brennan T . Analysis of errors reported by surgeons at three teaching hospitals. Surgery. 2003; 133(6):614-21. DOI: 10.1067/msy.2003.169. View

13.

Mishra A, Catchpole K, McCulloch P . The Oxford NOTECHS System: reliability and validity of a tool for measuring teamwork behaviour in the operating theatre. Qual Saf Health Care. 2009; 18(2):104-8. DOI: 10.1136/qshc.2007.024760. View

14.

Zhang T, Yang J, Liang N, Pitts B, Prakah-Asante K, Curry R . Physiological Measurements of Situation Awareness: A Systematic Review. Hum Factors. 2020; 65(5):737-758. DOI: 10.1177/0018720820969071. View

15.

Mishra A, Catchpole K, Dale T, McCulloch P . The influence of non-technical performance on technical outcome in laparoscopic cholecystectomy. Surg Endosc. 2007; 22(1):68-73. DOI: 10.1007/s00464-007-9346-1. View

16.

Cha J, Athanasiadis D, Anton N, Stefanidis D, Yu D . Measurement of Nontechnical Skills During Robotic-Assisted Surgery Using Sensor-Based Communication and Proximity Metrics. JAMA Netw Open. 2021; 4(11):e2132209. PMC: 8564574. DOI: 10.1001/jamanetworkopen.2021.32209. View

17.

Downing S . Threats to the validity of clinical teaching assessments: what about rater error?. Med Educ. 2005; 39(4):353-5. DOI: 10.1111/j.1365-2929.2005.02138.x. View

18.

Lee K, Vaishnavi S, Lau S, Andriole D, Jeffe D . Cultural competency in medical education: demographic differences associated with medical student communication styles and clinical clerkship feedback. J Natl Med Assoc. 2009; 101(2):116-26. DOI: 10.1016/s0027-9684(15)30823-3. View

19.

Wu C, Cha J, Sulek J, Zhou T, Sundaram C, Wachs J . Eye-Tracking Metrics Predict Perceived Workload in Robotic Surgical Skills Training. Hum Factors. 2019; 62(8):1365-1386. PMC: 7672675. DOI: 10.1177/0018720819874544. View

20.

Cha J, Anton N, Mizota T, Hennings J, Rendina M, Stanton-Maxey K . Use of non-technical skills can predict medical student performance in acute care simulated scenarios. Am J Surg. 2018; 217(2):323-328. DOI: 10.1016/j.amjsurg.2018.09.028. View