Evaluating Pre-anesthesia Assessment Performance in Residency: the Reliability of Standardized Patient Methods

Overview

Journal Front Med (Lausanne)

Specialty General Medicine

Date 2024 Aug 16

PMID 39149611

Authors

Emmanuel Besnier

Sebastien Franchina

Antoine Lefevre-Scelles

Thierry Wable

Jean-Luc Hanouz

Etienne Allard

Bertrand Dureuil

Vincent Compere

Affiliations

Soon will be listed here.

Abstract

Background: Pre-anesthesia assessment clinic (PAC) is known to increase safety and quality in the perioperative period. However, PAC teaching during anesthesiology residency is a challenge. The objective of this study was to assess the reliability of a simulation score grid using a standardized patient on the PAC performance of anesthesiology residents.

Methods: A score grid, including the 4 components of the PAC (clinical evaluation, perioperative strategy, information and communication) was validated by a group of 5 senior anesthesiologists. Anesthesiology residents (> one year) and attending anesthesiologists were included. The same simulation sequence with the same standardized patient was conducted in a simulation dedicated consultation room. The simulation sequence was followed by a debriefing session with the 2 professors (anesthesiology and communication) and each anesthesiology resident. The main outcome was the overall grid score out of a maximum score of 300 and the correlation of this score with experience in anesthesiology residency. Secondary outcomes were individual component scores according to level of experience in anesthesiology.

Results: Between October 2014 and April 2016, 109 anesthesiology residents and 16 attending anesthesiologists were included in this prospective bicentric study. There was a positive correlation ( < 0.01) between level of experience and overall score on the grid score (Pearson's Coefficient = 0.52). The Pearson correlation coefficient between overall assessment and level of experience in anesthesiology was 0.46 ( < 0.01). The analysis of the sub-scores for the 4 components of the overall score (evaluation, perioperative strategy, information and communication) also identify differences between groups of experience.

Conclusion: Standardized patient Simulation of PAC seems to be a reliable tool to assess PAC performance in anesthesiology residents and senior anesthesiologists. These results suggest standardized patient simulation could be used as a teaching tool for PAC.

References

Kluger M, Tham E, Coleman N, Runciman W, Bullock M . Inadequate pre-operative evaluation and preparation: a review of 197 reports from the Australian incident monitoring study. Anaesthesia. 2000; 55(12):1173-8. DOI: 10.1046/j.1365-2044.2000.01725.x. View

Setyonugroho W, Kennedy K, Kropmans T . Reliability and validity of OSCE checklists used to assess the communication skills of undergraduate medical students: A systematic review. Patient Educ Couns. 2015; . DOI: 10.1016/j.pec.2015.06.004. View

Harden R, Stevenson M, Downie W, WILSON G . Assessment of clinical competence using objective structured examination. Br Med J. 1975; 1(5955):447-51. PMC: 1672423. DOI: 10.1136/bmj.1.5955.447. View

Compere V, Froemer B, Clavier T, Selim J, Burey J, Dureuil B . Evaluation of the Duration of Preanesthesia Consultation: Prospective and Multicenter Study. Anesth Analg. 2022; 134(3):496-504. DOI: 10.1213/ANE.0000000000005889. View

Soltner C, Giquello J, Monrigal-Martin C, Beydon L . Continuous care and empathic anaesthesiologist attitude in the preoperative period: impact on patient anxiety and satisfaction. Br J Anaesth. 2011; 106(5):680-6. DOI: 10.1093/bja/aer034. View

Pell G, Fuller R, Homer M, Roberts T . How to measure the quality of the OSCE: A review of metrics - AMEE guide no. 49. Med Teach. 2010; 32(10):802-11. DOI: 10.3109/0142159X.2010.507716. View

Khan K, Gaunt K, Ramachandran S, Pushkar P . The Objective Structured Clinical Examination (OSCE): AMEE Guide No. 81. Part II: organisation & administration. Med Teach. 2013; 35(9):e1447-63. DOI: 10.3109/0142159X.2013.818635. View

OSullivan P, Chao S, Russell M, Levine S, Fabiny A . Development and implementation of an objective structured clinical examination to provide formative feedback on communication and interpersonal skills in geriatric training. J Am Geriatr Soc. 2008; 56(9):1730-5. DOI: 10.1111/j.1532-5415.2008.01860.x. View

Daniels V, Pugh D . Twelve tips for developing an OSCE that measures what you want. Med Teach. 2017; 40(12):1208-1213. DOI: 10.1080/0142159X.2017.1390214. View

10.

Piumatti G, Cerutti B, Perron N . Assessing communication skills during OSCE: need for integrated psychometric approaches. BMC Med Educ. 2021; 21(1):106. PMC: 7887794. DOI: 10.1186/s12909-021-02552-8. View

11.

Haskins I, Amdur R, Vaziri K . The effect of smoking on bariatric surgical outcomes. Surg Endosc. 2014; 28(11):3074-80. DOI: 10.1007/s00464-014-3581-z. View

12.

de Oliveira Filho G, Schonhorst L . The development and application of an instrument for assessing resident competence during preanesthesia consultation. Anesth Analg. 2004; 99(1):62-69. DOI: 10.1213/01.ANE.0000121770.64938.3B. View

13.

Lukas R, Adesoye T, Smith S, Blood A, Brorson J . Student assessment by objective structured examination in a neurology clerkship. Neurology. 2012; 79(7):681-5. PMC: 3414663. DOI: 10.1212/WNL.0b013e3182648ba1. View

14.

Hawkins R, Swanson B, Kremer M . An integrative review of factors related to patient satisfaction with general anesthesia care. AORN J. 2012; 96(4):368-76. DOI: 10.1016/j.aorn.2012.07.015. View

15.

Bondy L, Sims N, Schroeder D, Offord K, Narr B . The effect of anesthetic patient education on preoperative patient anxiety. Reg Anesth Pain Med. 1999; 24(2):158-64. DOI: 10.1016/s1098-7339(99)90078-0. View

16.

Lozada M, Nguyen J, Abouleish A, Prough D, Przkora R . Patient preference for the pre-anesthesia evaluation: Telephone versus in-office assessment. J Clin Anesth. 2016; 31:145-8. DOI: 10.1016/j.jclinane.2015.12.040. View

17.

Muca E, Molino M, Ghislieri C, Baratta M, Odore R, Bergero D . Relationships between psychological characteristics, academic fit and engagement with academic performance in veterinary medical students. BMC Vet Res. 2023; 19(1):132. PMC: 10463963. DOI: 10.1186/s12917-023-03695-0. View

18.

Barneschi M, Miccinesi G, Marini F, Bressan F, Paci E . Informing patients about risks and complications of anaesthesia. Minerva Anestesiol. 2003; 68(11):811-8, 818-23. View

19.

Harms C, Young J, Amsler F, Zettler C, Scheidegger D, Kindler C . Improving anaesthetists' communication skills. Anaesthesia. 2004; 59(2):166-72. DOI: 10.1111/j.1365-2044.2004.03528.x. View

20.

Nasir A, Yusuf A, Abdur-Rahman L, Babalola O, Adeyeye A, Popoola A . Medical students' perception of objective structured clinical examination: a feedback for process improvement. J Surg Educ. 2014; 71(5):701-6. DOI: 10.1016/j.jsurg.2014.02.010. View