Development of a Competency Assessment Framework for Pharmacy Residents Participating in High-Fidelity Simulation and Creation of the SIMPHARM Assessment Tool

Overview

Journal Can J Hosp Pharm

Date 2024 Jul 11

PMID 38988872

Authors

Patrick Jf Deschenes

My Hanh Luu Hoai

Justine Rinfret

Sarah Pelletier

Alessandra Stortini

Rachel Therrien

Marie-Kim Heraut

Pascaline Bernier

John Nam-Kha Nguyen

Affiliations

Soon will be listed here.

Abstract

Background: Simulations are used to improve professional practice across multiple health professions; however, the application of simulations in pharmacy curricula has been limited by a lack of evidence.

Objectives: To delineate the competencies of pharmacy residents needed to assess their progress while participating in a high-fidelity simulation training program and to create a reliable assessment tool based on these competencies.

Methods: A literature review was conducted, followed by application of a modified Delphi method. An assessment tool based on the results of these activities was drafted (in French). A second modified Delphi process was carried out to assess the reliability of the tool, and the tool was modified accordingly.

Results: The literature search yielded a total of 2670 articles, of which 289 were included for analysis. The first modified Delphi process involved 19 experts in the first round and 10 experts in the second round. The Cronbach α was 0.866 (95% confidence interval [CI] 0.713-0.960), indicating good reliability. A total of 7 competencies were retained for inclusion in the SIMPHARM assessment tool: professionalism, leadership, teamwork, communication, critical thinking, preparation and packaging of medications, and pharmaceutical calculations. The second modified Delphi process involved 9 experts in the first round and 11 experts in the second round. The final Cronbach α was 0.877 (95% CI 0.741-0.960).

Conclusions: To the authors' knowledge, this was one of the largest studies exploring pharmacy competencies in the context of simulations. This work yielded a reliable framework for the assessment of pharmacy residents' competencies. This assessment tool may help evaluators in assessing the competencies of pharmacy trainees after simulation training.

References

Issenberg S, McGaghie W, Petrusa E, Gordon D, Scalese R . Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Med Teach. 2005; 27(1):10-28. DOI: 10.1080/01421590500046924. View

Bray B, Schwartz C, Odegard P, Hammer D, Seybert A . Assessment of human patient simulation-based learning. Am J Pharm Educ. 2012; 75(10):208. PMC: 3279041. DOI: 10.5688/ajpe7510208. View

Barnett S, Allen S, Bastianelli K, Chen J, Clark Dula C, Djuric Kachlic M . Determination of Interrater Reliability of a Universal Evaluator Rubric to Assess Student Pharmacist Communication Skills. Am J Pharm Educ. 2021; 86(1):8544. PMC: 8787169. DOI: 10.5688/ajpe8544. View

Lasater K . Clinical judgment: the last frontier for evaluation. Nurse Educ Pract. 2011; 11(2):86-92. DOI: 10.1016/j.nepr.2010.11.013. View

Kane-Gill S, Smithburger P . Transitioning knowledge gained from simulation to pharmacy practice. Am J Pharm Educ. 2012; 75(10):210. PMC: 3279027. DOI: 10.5688/ajpe7510210. View

Robinson J, Bray B, Willson M, Weeks D . Using human patient simulation to prepare student pharmacists to manage medical emergencies in an ambulatory setting. Am J Pharm Educ. 2011; 75(1):3. PMC: 3049662. DOI: 10.5688/ajpe7513. View

Ford D, Seybert A, Smithburger P, Kobulinsky L, Samosky J, Kane-Gill S . Impact of simulation-based learning on medication error rates in critically ill patients. Intensive Care Med. 2010; 36(9):1526-31. DOI: 10.1007/s00134-010-1860-2. View

Tavakol M, Dennick R . Making sense of Cronbach's alpha. Int J Med Educ. 2016; 2:53-55. PMC: 4205511. DOI: 10.5116/ijme.4dfb.8dfd. View

Kim J, Neilipovitz D, Cardinal P, Chiu M, Clinch J . A pilot study using high-fidelity simulation to formally evaluate performance in the resuscitation of critically ill patients: The University of Ottawa Critical Care Medicine, High-Fidelity Simulation, and Crisis Resource Management I Study. Crit Care Med. 2006; 34(8):2167-74. DOI: 10.1097/01.CCM.0000229877.45125.CC. View

10.

Schroeder M, Lengel A . Evaluation of a Rubric to Assess Pharmacy Student Performance in a Telehealth Simulation Exercise. Am J Pharm Educ. 2021; 86(8):ajpe8778. PMC: 10159385. DOI: 10.5688/ajpe8778. View

11.

Yule S, Flin R, Paterson-Brown S, Maran N, Rowley D . Development of a rating system for surgeons' non-technical skills. Med Educ. 2006; 40(11):1098-104. DOI: 10.1111/j.1365-2929.2006.02610.x. View

12.

De Villiers M, de Villiers P, Kent A . The Delphi technique in health sciences education research. Med Teach. 2005; 27(7):639-43. DOI: 10.1080/13611260500069947. View

13.

Bailey L, Curington R, Brown B, Hegener M, Espel M . Motivational interviewing education: Creation and assessment of a learning module implemented among advanced pharmacy practice students. Curr Pharm Teach Learn. 2017; 9(5):786-793. DOI: 10.1016/j.cptl.2017.06.007. View

14.

Bujang M, Omar E, Baharum N . A Review on Sample Size Determination for Cronbach's Alpha Test: A Simple Guide for Researchers. Malays J Med Sci. 2019; 25(6):85-99. PMC: 6422571. DOI: 10.21315/mjms2018.25.6.9. View

15.

Bravo G, Potvin L . Estimating the reliability of continuous measures with Cronbach's alpha or the intraclass correlation coefficient: toward the integration of two traditions. J Clin Epidemiol. 1991; 44(4-5):381-90. DOI: 10.1016/0895-4356(91)90076-l. View

16.

Martin J, Regehr G, Reznick R, MacRae H, Murnaghan J, Hutchison C . Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg. 1997; 84(2):273-8. DOI: 10.1046/j.1365-2168.1997.02502.x. View

17.

Vyas D, Wombwell E, Russell E, Caligiuri F . High-fidelity patient simulation series to supplement introductory pharmacy practice experiences. Am J Pharm Educ. 2011; 74(9):169. PMC: 2996759. DOI: 10.5688/aj7409169. View

18.

Doloresco F, Maerten-Rivera J, Zhao Y, Foltz-Ramos K, Fusco N . Pharmacy Students' Standardized Self-Assessment of Interprofessional Skills During an Objective Structured Clinical Examination. Am J Pharm Educ. 2020; 83(10):7439. PMC: 6983892. DOI: 10.5688/ajpe7439. View

19.

Bajis D, Chaar B, Basheti I, Moles R . Pharmacy students' medication history taking competency: Simulation and feedback learning intervention. Curr Pharm Teach Learn. 2019; 11(10):1002-1015. DOI: 10.1016/j.cptl.2019.06.007. View

20.

Flin R, Patey R . Non-technical skills for anaesthetists: developing and applying ANTS. Best Pract Res Clin Anaesthesiol. 2011; 25(2):215-27. DOI: 10.1016/j.bpa.2011.02.005. View