Postprocedural Cognitive Load Measurement With Immediate Feedback to Guide Curriculum Development

Overview

Journal Ann Thorac Surg

Publisher Elsevier

Specialties General Surgery
Pulmonary Medicine

Date 2021 Jul 2

PMID 34214548

Citations 1

Authors

Lauren V Huckaby

Anthony R Cyr

Robert M Handzel

Eliza Beth Littleton

Lawrence R Crist

James D Luketich

Kenneth K Lee

Rajeev Dhupar

Affiliations

Soon will be listed here.

Abstract

Background: Methods to assess competency in cardiothoracic training are essential. Here, we report a system that allows us to better assess competency from the perspective of both the trainee and educator. We hypothesized that postprocedural cognitive burden measurement (by the trainee) with immediate feedback (from the educator) could aid in identifying barriers to the acquisition of skills and knowledge so that training curricula can be individualized.

Methods: The National Aeronautics and Space Administration Task Load Index (NASA-TLX), a validated instrument to measure cognitive load, was administered with an online platform after bronchoscopy, esophagogastroduodenoscopy, and video-assisted thoracoscopic surgery for 11 residents. Immediate postprocedure feedback and standardized debriefing occurred for each procedure.

Results: Mean NASA-TLX scores were highest (indicating greater cognitive load) for esophagogastroduodenoscopy and video-assisted thoracoscopic surgery (P < .001). When comparing subscale measures, mental demand was significantly higher for video-assisted thoracoscopic surgery (P = .026) compared with the other procedures, whereas physical demand was highest for esophagogastroduodenoscopy (P = .018). Self-reported frustration was similar for all case types (P = .247). Cognitive burden decreased with a greater number of procedures for bronchoscopy (P = .027). Significant improvement was noted by the trainee at the end of the rotation in self-assessed procedural competency and preparedness for thoracic board topics (all P < .05). Postprocedure feedback by the attending surgeon correlated with more frequent completion of self-evaluations by the residents.

Conclusions: Longitudinal assessment of cognitive load in combination with postprocedural feedback identified barriers to skill acquisition for both residents and educators. This information allows for individualized rotation development as a step toward a competency-based curriculum.

Citing Articles

Relationship Between Cognitive Load Theory, Intrinsic Motivation and Emotions in Healthcare Professions Education: A Perspective on the Missing Link.

Patel D, Alismail A Adv Med Educ Pract. 2024; 15:57-62.

PMID: 38264765 PMC: 10804965. DOI: 10.2147/AMEP.S441405.

References

Hayden E, Seagull F, Reddy R . Developing an educational video on lung lobectomy for the general surgery resident. J Surg Res. 2015; 196(2):216-20. DOI: 10.1016/j.jss.2015.02.020. View

Qiao Y, Shen J, Liang X, Ding S, Chen F, Shao L . Using cognitive theory to facilitate medical education. BMC Med Educ. 2014; 14:79. PMC: 3989791. DOI: 10.1186/1472-6920-14-79. View

Cook M, Watters J, Barton J, Kamin C, Brown S, Deveney K . A flexible postoperative debriefing process can effectively provide formative resident feedback. J Am Coll Surg. 2015; 220(5):959-67. DOI: 10.1016/j.jamcollsurg.2014.12.048. View

Schubert C, Denmark T, Crandall B, Grome A, Pappas J . Characterizing novice-expert differences in macrocognition: an exploratory study of cognitive work in the emergency department. Ann Emerg Med. 2012; 61(1):96-109. DOI: 10.1016/j.annemergmed.2012.08.034. View

Bilimoria K, Chung J, Hedges L, Dahlke A, Love R, Cohen M . National Cluster-Randomized Trial of Duty-Hour Flexibility in Surgical Training. N Engl J Med. 2016; 374(8):713-27. DOI: 10.1056/NEJMoa1515724. View

Bolster L, Rourke L . The Effect of Restricting Residents' Duty Hours on Patient Safety, Resident Well-Being, and Resident Education: An Updated Systematic Review. J Grad Med Educ. 2015; 7(3):349-63. PMC: 4597944. DOI: 10.4300/JGME-D-14-00612.1. View

Ragalie W, Termuhlen P, Little A . Changes in Thoracic Surgery Experience During General Surgery Residency: A Review of the Case Logs From the Accreditation Council for Graduate Medical Education. Ann Thorac Surg. 2016; 102(6):2095-2098. DOI: 10.1016/j.athoracsur.2016.06.058. View

Wagner J, Lewis C, Tillou A, Agopian V, Quach C, Donahue T . Use of Entrustable Professional Activities in the Assessment of Surgical Resident Competency. JAMA Surg. 2017; 153(4):335-343. PMC: 5933400. DOI: 10.1001/jamasurg.2017.4547. View

Porte M, Xeroulis G, Reznick R, Dubrowski A . Verbal feedback from an expert is more effective than self-accessed feedback about motion efficiency in learning new surgical skills. Am J Surg. 2006; 193(1):105-10. DOI: 10.1016/j.amjsurg.2006.03.016. View

10.

Dias R, Ngo-Howard M, Boskovski M, Zenati M, Yule S . Systematic review of measurement tools to assess surgeons' intraoperative cognitive workload. Br J Surg. 2018; 105(5):491-501. PMC: 5878696. DOI: 10.1002/bjs.10795. View

11.

DeBoard Z, Paisley M, Thomas D . Self-Appraised Readiness of Senior and Graduating General Surgery Residents to Perform Thoracic Surgery. J Surg Educ. 2017; 75(4):877-883. DOI: 10.1016/j.jsurg.2017.11.010. View

12.

Ruiz-Rabelo J, Navarro-Rodriguez E, Di-Stasi L, Diaz-Jimenez N, Cabrera-Bermon J, Diaz-Iglesias C . Validation of the NASA-TLX Score in Ongoing Assessment of Mental Workload During a Laparoscopic Learning Curve in Bariatric Surgery. Obes Surg. 2015; 25(12):2451-6. DOI: 10.1007/s11695-015-1922-1. View

13.

Yurko Y, Scerbo M, Prabhu A, Acker C, Stefanidis D . Higher mental workload is associated with poorer laparoscopic performance as measured by the NASA-TLX tool. Simul Healthc. 2011; 5(5):267-71. DOI: 10.1097/SIH.0b013e3181e3f329. View

14.

Friedell M, VanderMeer T, Cheatham M, Fuhrman G, Schenarts P, Mellinger J . Perceptions of graduating general surgery chief residents: are they confident in their training?. J Am Coll Surg. 2014; 218(4):695-703. DOI: 10.1016/j.jamcollsurg.2013.12.022. View

15.

Stride H, George B, Williams R, Bohnen J, Eaton M, Schuller M . Relationship of procedural numbers with meaningful procedural autonomy in general surgery residents. Surgery. 2017; 163(3):488-494. DOI: 10.1016/j.surg.2017.10.011. View

16.

Haji F, Rojas D, Childs R, De Ribaupierre S, Dubrowski A . Measuring cognitive load: performance, mental effort and simulation task complexity. Med Educ. 2015; 49(8):815-27. DOI: 10.1111/medu.12773. View

17.

Walsh C, Sherlock M, Ling S, Carnahan H . Virtual reality simulation training for health professions trainees in gastrointestinal endoscopy. Cochrane Database Syst Rev. 2012; (6):CD008237. DOI: 10.1002/14651858.CD008237.pub2. View

18.

van Merrienboer J, Sweller J . Cognitive load theory in health professional education: design principles and strategies. Med Educ. 2010; 44(1):85-93. DOI: 10.1111/j.1365-2923.2009.03498.x. View

19.

Williams R, Swanson D, Fryer J, Meyerson S, Bohnen J, Dunnington G . How Many Observations are Needed to Assess a Surgical Trainee's State of Operative Competency?. Ann Surg. 2017; 269(2):377-382. DOI: 10.1097/SLA.0000000000002554. View

20.

Liberman A, Liberman M, Steinert Y, McLeod P, Meterissian S . Surgery residents and attending surgeons have different perceptions of feedback. Med Teach. 2005; 27(5):470-2. DOI: 10.1080/0142590500129183. View