Remote Online Open Book Examinations: Through the Lenses of Faculty and Students in Health Professions Programmes

Overview

Journal BMC Med Educ

Publisher Biomed Central

Specialty Medical Education

Date 2023 Jun 2

PMID 37268906

Authors

Hui Meng Er

Pei Se Wong

Vishna Devi Nadarajah

Affiliations

Soon will be listed here.

Abstract

Background: Open book examinations have been used to assess students' higher order cognitive skills. These examinations can be conducted online remotely with the advancement of technology. However, there are concerns regarding its validity and reliability particularly if the examinations are not proctored. The objective of this study was to explore the perceptions of faculty and students in health professions programmes about remote online open book examinations (ROOBE).

Methods: Semi-structured interviews were conducted among 22 faculty staff who were involved in ROOBE in health professions programmes. All interviews were audio recorded, transcribed verbatim and analysed using a thematic analysis approach. The perceptions of 249 medical students were obtained using an online questionnaire after they completed ROOBE.

Results: The faculty agreed that open book examinations could promote students' higher order cognitive skills and reduce students' stress. However, they were concerned about students' academic integrity during non-invigilated ROOBE which could affect recognition by accreditation and professional bodies. The shift from traditional practice of closed-book examinations to ROOBE required change management with the support of guidelines and faculty training. Majority of the students claimed that the examinations were challenging as they assessed their ability to apply knowledge in real world problems. Nevertheless, they preferred ROOBE due to less anxiety and memorisation, and more emphasis on problem solving skills. The shortcomings were insufficient time for information searching during examinations and uncertainty in preparedness for future practice as they focused less on memorisation of factual knowledge during examination preparation. Cheating among peers and internet instability during non-invigilated ROOBE were the concerns highlighted by some students.

Conclusions: Faculty and students expressed favourable views about ROOBE in promoting higher order cognitive skills. Adequate technological support was essential during ROOBE. While there was a need to address issues related to academic integrity, ROOBE could be included as an authentic assessment within the systems of assessment.

Citing Articles

STEM exam performance: Open- versus closed-book methods in the large language model era.

Mizori R, Sadiq M, Ahmad M, Siu A, Ahmad R, Yang Z Clin Teach. 2024; 22(1):e13839.

PMID: 39496553 PMC: 11663729. DOI: 10.1111/tct.13839.

References

Heijne-Penninga M, Kuks J, Hofman W, Cohen-Schotanus J . Influence of open- and closed-book tests on medical students' learning approaches. Med Educ. 2008; 42(10):967-74. DOI: 10.1111/j.1365-2923.2008.03125.x. View

Pandey P, Zimitat C . Medical students' learning of anatomy: memorisation, understanding and visualisation. Med Educ. 2007; 41(1):7-14. DOI: 10.1111/j.1365-2929.2006.02643.x. View

Bughrara M, Swanberg S, Lucia V, Schmitz K, Jung D, Wunderlich-Barillas T . Beyond COVID-19: the impact of recent pandemics on medical students and their education: a scoping review. Med Educ Online. 2022; 28(1):2139657. PMC: 9639463. DOI: 10.1080/10872981.2022.2139657. View

Bladt F, Khanal P, Prabhu A, Hauke E, Kingsbury M, Saleh S . Medical students' perception of changes in assessments implemented during the COVID-19 pandemic. BMC Med Educ. 2022; 22(1):844. PMC: 9727955. DOI: 10.1186/s12909-022-03787-9. View

Nigam A, Pasricha R, Singh T, Churi P . A Systematic Review on AI-based Proctoring Systems: Past, Present and Future. Educ Inf Technol (Dordr). 2021; 26(5):6421-6445. PMC: 8220875. DOI: 10.1007/s10639-021-10597-x. View

Van Der Vleuten C . The assessment of professional competence: Developments, research and practical implications. Adv Health Sci Educ Theory Pract. 2013; 1(1):41-67. DOI: 10.1007/BF00596229. View

Broyles I, Cyr P, Korsen N . Open book tests: assessment of academic learning in clerkships. Med Teach. 2005; 27(5):456-62. DOI: 10.1080/01421590500097075. View

DiCarlo S . Too much content, not enough thinking, and too little fun!. Adv Physiol Educ. 2009; 33(4):257-64. DOI: 10.1152/advan.00075.2009. View

Davies D, McLean P, Kemp P, Liddle A, Morrell M, Halse O . Assessment of factual recall and higher-order cognitive domains in an open-book medical school examination. Adv Health Sci Educ Theory Pract. 2021; 27(1):147-165. PMC: 8536902. DOI: 10.1007/s10459-021-10076-5. View

10.

Norcini J, Anderson M, Bollela V, Burch V, Costa M, Duvivier R . 2018 Consensus framework for good assessment. Med Teach. 2018; 40(11):1102-1109. DOI: 10.1080/0142159X.2018.1500016. View

11.

Al-Sharbatti S, Hamdy H, Kassab S, Venkatramana M . The effect of access to electronic resources during examination on medical and dental students scores in summative assessment: quasi experimental study. BMC Med Educ. 2022; 22(1):859. PMC: 9744029. DOI: 10.1186/s12909-022-03904-8. View

12.

Durning S, Dong T, Ratcliffe T, Schuwirth L, Artino Jr A, Boulet J . Comparing Open-Book and Closed-Book Examinations: A Systematic Review. Acad Med. 2015; 91(4):583-99. DOI: 10.1097/ACM.0000000000000977. View

13.

Ramamurthy S, Er H, Nadarajah V, Pook P . Study on the impact of open and closed book formative examinations on pharmacy students' performance, perception, and learning approach. Curr Pharm Teach Learn. 2018; 8(3):364-374. DOI: 10.1016/j.cptl.2016.02.017. View

14.

Bohay M, Blakely D, Tamplin A, Radvansky G . Note taking, review, memory, and comprehension. Am J Psychol. 2011; 124(1):63-73. DOI: 10.5406/amerjpsyc.124.1.0063. View

15.

Teodorczuk A, Fraser J, Rogers G . Open book exams: A potential solution to the "full curriculum"?. Med Teach. 2017; 40(5):529-530. DOI: 10.1080/0142159X.2017.1412412. View