Familiarity with ChatGPT Features Modifies Expectations and Learning Outcomes of Dental Students

Objectives: The number of approvals for AI-based systems is increasing rapidly, although AI clinical trial designs lack consideration of the impact of human-AI interaction. Aim of this work was to investigate how reading of an AI system (ChatGPT) features/descriptions could influence the willingness and expectations for use of this technology as well as dental students' learning performance.

Methods: Dental students (N = 104) were asked to learn about side effects of drugs used in dental practice via reading recommended literature or ChatGPT. Expectations towards ChatGPT were measured by survey, before and after reading of a system features description, whilst learning outcomes were evaluated via pharmacology quiz.

Results: Students who used ChatGPT (YG group) showed better results on the pharmacology quiz than students who neither read the description nor employed ChatGPT for learning (NN condition). Moreover, students who read the description of ChatGPT features yet did not use it (NG) showed better results on the pharmacology quiz compared with the NN condition, although none of them employed ChatGPT for learning. The NG students compared to the YG students had less trust in AI system assistance in learning, and after the AI system description reading, their expectations changed significantly, showing an association with quiz scores.

Conclusions: A majority of students in our cohort was reluctant to use ChatGPT. Furthermore, familarity (reading) with ChatGPT features appear to alter the expectations and enhance learning performance of students.suggesting an AI description-related cognitive bias. Hence the content description of ChatGPTshould be reviewed and verified prior to AI system use for educational purposes.