Comparing the Use of DynaMed and UpToDate by Physician Trainees in Clinical Decision-Making: A Randomized Crossover Trial

Overview

Journal Appl Clin Inform

Publisher Thieme

Specialties Health Services
Medical Informatics

Date 2022 Feb 2

PMID 35108739

Authors

Sally L Baxter

Lina Lander

Brian Clay

John Bell

Kristen Hansen

Amanda Walker

Ming Tai-Seale

Affiliations

Soon will be listed here.

Abstract

Background: Costs vary substantially among electronic medical knowledge resources used for clinical decision support, warranting periodic assessment of institution-wide adoption.

Objectives: To compare two medical knowledge resources, UpToDate and DynaMed Plus, regarding accuracy and time required to answer standardized clinical questions and user experience.

Methods: A crossover trial design was used, wherein physicians were randomized to first use one of the two medical knowledge resources to answer six standardized questions. Following use of each resource, they were surveyed regarding their user experience. The percentage of accurate answers and time required to answer each question were recorded. The surveys assessed ease of use, enjoyment using the resource, quality of information, and ability to assess level of evidence. Tests of carry-over effects were performed. Themes were identified within open-ended survey comments regarding overall user experience.

Results: Among 26 participating physicians, accuracy of answers differed by 4 percentage points or less. For all but one question, there were no significant differences in the time required for completion. Most participants felt both resources were easy to use, contained high quality of information, and enabled assessment of the level of evidence. A greater proportion of participants endorsed enjoyment of use with UpToDate (23/26, 88%) compared with DynaMed Plus (16/26, 62%). Themes from open-ended comments included interface/information presentation, coverage of clinical topics, search functions, and utility for clinical decision-making. The majority (59%) of open-ended comments expressed an overall preference for UpToDate, compared with 19% preferring DynaMed Plus.

Conclusion: DynaMed Plus is noninferior to UpToDate with respect to ability to achieve accurate answers, time required for answering clinical questions, ease of use, quality of information, and ability to assess level of evidence. However, user experience was more positive with UpToDate. Future studies of electronic medical knowledge resources should continue to emphasize evaluation of usability and user experience.

Citing Articles

Understanding and training for the impact of large language models and artificial intelligence in healthcare practice: a narrative review.

McCoy L, Yu Ci Ng F, Sauer C, Yap Legaspi K, Jain B, Gallifant J BMC Med Educ. 2024; 24(1):1096.

PMID: 39375721 PMC: 11459854. DOI: 10.1186/s12909-024-06048-z.

Straight to the point: evaluation of a Point of Care Information (POCI) resource in answering disease-related questions.

Wasserman R, Seger D, Amato M, Co Z, Mugal A, Rui A J Med Libr Assoc. 2024; 112(1):13-21.

PMID: 38911524 PMC: 11189136. DOI: 10.5195/jmla.2024.1770.

References

Jeffery R, Navarro T, Lokker C, Haynes R, Wilczynski N, Farjou G . How current are leading evidence-based medical textbooks? An analytic survey of four online textbooks. J Med Internet Res. 2012; 14(6):e175. PMC: 3799557. DOI: 10.2196/jmir.2105. View

Bennett N, Casebeer L, Kristofco R, Strasser S . Physicians' Internet information-seeking behaviors. J Contin Educ Health Prof. 2004; 24(1):31-8. DOI: 10.1002/chp.1340240106. View

Alper B, White D, Ge B . Physicians answer more clinical questions and change clinical decisions more often with synthesized evidence: a randomized trial in primary care. Ann Fam Med. 2005; 3(6):507-13. PMC: 1466938. DOI: 10.1370/afm.370. View

Kumaravel B, Hearn J, Jahangiri L, Pollard R, Stocker C, Nunan D . A systematic review and taxonomy of tools for evaluating evidence-based medicine teaching in medical education. Syst Rev. 2020; 9(1):91. PMC: 7183115. DOI: 10.1186/s13643-020-01311-y. View

Shurtz S, Foster M . Developing and using a rubric for evaluating evidence-based medicine point-of-care tools. J Med Libr Assoc. 2011; 99(3):247-54. PMC: 3133902. DOI: 10.3163/1536-5050.99.3.012. View

Ebell M . How to find answers to clinical questions. Am Fam Physician. 2009; 79(4):293-6. View

Maggio L, Tannery N, Chen H, Ten Cate O, OBrien B . Evidence-based medicine training in undergraduate medical education: a review and critique of the literature published 2006-2011. Acad Med. 2013; 88(7):1022-8. DOI: 10.1097/ACM.0b013e3182951959. View

Addison J, Whitcombe J, Glover S . How doctors make use of online, point-of-care clinical decision support systems: a case study of UpToDate©. Health Info Libr J. 2013; 30(1):13-22. DOI: 10.1111/hir.12002. View

Dwan K, Li T, Altman D, Elbourne D . CONSORT 2010 statement: extension to randomised crossover trials. BMJ. 2019; 366:l4378. PMC: 6667942. DOI: 10.1136/bmj.l4378. View

10.

Holden R, Karsh B . The technology acceptance model: its past and its future in health care. J Biomed Inform. 2009; 43(1):159-72. PMC: 2814963. DOI: 10.1016/j.jbi.2009.07.002. View

11.

Johnson E, Emani V, Ren J . Breadth of Coverage, Ease of Use, and Quality of Mobile Point-of-Care Tool Information Summaries: An Evaluation. JMIR Mhealth Uhealth. 2016; 4(4):e117. PMC: 5081478. DOI: 10.2196/mhealth.6189. View

12.

OCarroll A, Westby E, Dooley J, Gordon K . Information-Seeking Behaviors of Medical Students: A Cross-Sectional Web-Based Survey. JMIR Med Educ. 2016; 1(1):e4. PMC: 5041342. DOI: 10.2196/mededu.4267. View

13.

Marshall J, Sollenberger J, Easterby-Gannett S, Morgan L, Klem M, Cavanaugh S . The value of library and information services in patient care: results of a multisite study. J Med Libr Assoc. 2013; 101(1):38-46. PMC: 3543128. DOI: 10.3163/1536-5050.101.1.007. View

14.

Loda T, Erschens R, Junne F, Stengel A, Zipfel S, Herrmann-Werner A . Undergraduate Medical Students' Search for Health Information Online: Explanatory Cross-Sectional Study. JMIR Med Inform. 2020; 8(3):e16279. PMC: 7076408. DOI: 10.2196/16279. View

15.

Bradley-Ridout G, Nekolaichuk E, Jamieson T, Jones C, Morson N, Chuang R . UpToDate versus DynaMed: a cross-sectional study comparing the speed and accuracy of two point-of-care information tools. J Med Libr Assoc. 2021; 109(3):382-387. PMC: 8485969. DOI: 10.5195/jmla.2021.1176. View

16.

Walden R, Woodward N, Wallace R . Reevaluating Point-of-Care Resources: Community Engagement in Difficult Collection Choices. Med Ref Serv Q. 2019; 38(1):22-30. DOI: 10.1080/02763869.2019.1547055. View

17.

Scaffidi M, Khan R, Wang C, Keren D, Tsui C, Garg A . Comparison of the Impact of Wikipedia, UpToDate, and a Digital Textbook on Short-Term Knowledge Acquisition Among Medical Students: Randomized Controlled Trial of Three Web-Based Resources. JMIR Med Educ. 2017; 3(2):e20. PMC: 5686416. DOI: 10.2196/mededu.8188. View

18.

Charbonneau D, James L . DynaMed Plus: An Evidence-Based Clinical Reference Resource. Med Ref Serv Q. 2018; 37(2):168-176. DOI: 10.1080/02763869.2018.1439221. View

19.

Ellsworth M, Homan J, Cimino J, Peters S, Pickering B, Herasevich V . Point-of-Care Knowledge-Based Resource Needs of Clinicians: A Survey from a Large Academic Medical Center. Appl Clin Inform. 2015; 6(2):305-17. PMC: 4493332. DOI: 10.4338/ACI-2014-11-RA-0104. View

20.

Maggio L, Aakre C, Del Fiol G, Shellum J, Cook D . Impact of Clinicians' Use of Electronic Knowledge Resources on Clinical and Learning Outcomes: Systematic Review and Meta-Analysis. J Med Internet Res. 2019; 21(7):e13315. PMC: 6690166. DOI: 10.2196/13315. View