Sensitivity and Predictive Value of 15 PubMed Search Strategies to Answer Clinical Questions Rated Against Full Systematic Reviews

Overview

Journal J Med Internet Res

Publisher JMIR Publications

Specialty Medical Informatics

Date 2012 Jun 14

PMID 22693047

Citations 20

Authors

Thomas Agoritsas

Arnaud Merglen

Delphine S Courvoisier

Christophe Combescure

Nicolas Garin

Arnaud Perrier

Thomas V Perneger

Affiliations

Soon will be listed here.

Abstract

Background: Clinicians perform searches in PubMed daily, but retrieving relevant studies is challenging due to the rapid expansion of medical knowledge. Little is known about the performance of search strategies when they are applied to answer specific clinical questions.

Objective: To compare the performance of 15 PubMed search strategies in retrieving relevant clinical trials on therapeutic interventions.

Methods: We used Cochrane systematic reviews to identify relevant trials for 30 clinical questions. Search terms were extracted from the abstract using a predefined procedure based on the population, interventions, comparison, outcomes (PICO) framework and combined into queries. We tested 15 search strategies that varied in their query (PIC or PICO), use of PubMed's Clinical Queries therapeutic filters (broad or narrow), search limits, and PubMed links to related articles. We assessed sensitivity (recall) and positive predictive value (precision) of each strategy on the first 2 PubMed pages (40 articles) and on the complete search output.

Results: The performance of the search strategies varied widely according to the clinical question. Unfiltered searches and those using the broad filter of Clinical Queries produced large outputs and retrieved few relevant articles within the first 2 pages, resulting in a median sensitivity of only 10%-25%. In contrast, all searches using the narrow filter performed significantly better, with a median sensitivity of about 50% (all P < .001 compared with unfiltered queries) and positive predictive values of 20%-30% (P < .001 compared with unfiltered queries). This benefit was consistent for most clinical questions. Searches based on related articles retrieved about a third of the relevant studies.

Conclusions: The Clinical Queries narrow filter, along with well-formulated queries based on the PICO framework, provided the greatest aid in retrieving relevant clinical trials within the 2 first PubMed pages. These results can help clinicians apply effective strategies to answer their questions at the point of care.

Citing Articles

Added Value of Medical Subject Headings Terms in Search Strategies of Systematic Reviews: Comparative Study.

Leblanc V, Hamroun A, Bentegeac R, Le Guellec B, Lenain R, Chazard E J Med Internet Res. 2024; 26:e53781.

PMID: 39561364 PMC: 11615561. DOI: 10.2196/53781.

Sentinel Lymph Node Detection in Cutaneous Melanoma Using Indocyanine Green-Based Near-Infrared Fluorescence Imaging: A Systematic Review and Meta-Analysis.

Wolffer M, Liechti R, Constantinescu M, Lese I, Zubler C Cancers (Basel). 2024; 16(14).

PMID: 39061163 PMC: 11274776. DOI: 10.3390/cancers16142523.

The safety and efficacy of direct oral anticoagulants among chronic kidney disease patients on dialysis with non-valvular atrial fibrillation: a meta-analysis.

Mapili J, Lim L, Velando B, Aherrera J Front Cardiovasc Med. 2023; 10:1261183.

PMID: 37795477 PMC: 10545858. DOI: 10.3389/fcvm.2023.1261183.

Collagenase clostridium histolyticum injection versus limited fasciectomy for the treatment of Dupuytren's disease: a systematic review and meta-analysis of comparative studies.

Liechti R, Merky D, Sutter D, Ipaktchi R, Vogelin E Arch Orthop Trauma Surg. 2023; 144(1):527-536.

PMID: 37665353 DOI: 10.1007/s00402-023-05004-8.

Systematic Review on Individualized Versus Standardized Parenteral Nutrition in Preterm Infants.

Mihatsch W, Jimenez Varas M, Diehl L, Carnielli V, Schuler R, Gebauer C Nutrients. 2023; 15(5).

PMID: 36904223 PMC: 10005430. DOI: 10.3390/nu15051224.

References

Hoogendam A, Stalenhoef A, de Vries Robbe P, Overbeke A . Answers to questions posed during daily patient care are more likely to be answered by UpToDate than PubMed. J Med Internet Res. 2008; 10(4):e29. PMC: 2629368. DOI: 10.2196/jmir.1012. View

Green M, Ciampi M, Ellis P . Residents' medical information needs in clinic: are they being met?. Am J Med. 2000; 109(3):218-23. DOI: 10.1016/s0002-9343(00)00458-7. View

Shariff S, Bejaimal S, Sontrop J, Iansavichus A, Weir M, Haynes R . Searching for medical information online: a survey of Canadian nephrologists. J Nephrol. 2011; 24(6):723-32. DOI: 10.5301/JN.2011.6373. View

Thiele R, Poiro N, Scalzo D, Nemergut E . Speed, accuracy, and confidence in Google, Ovid, PubMed, and UpToDate: results of a randomised trial. Postgrad Med J. 2010; 86(1018):459-65. DOI: 10.1136/pgmj.2010.098053. View

Sampson M, Zhang L, Morrison A, Barrowman N, Clifford T, Platt R . An alternative to the hand searching gold standard: validating methodological search filters using relative recall. BMC Med Res Methodol. 2006; 6:33. PMC: 1557524. DOI: 10.1186/1471-2288-6-33. View

McKibbon K, Fridsma D . Effectiveness of clinician-selected electronic information resources for answering primary care physicians' information needs. J Am Med Inform Assoc. 2006; 13(6):653-9. PMC: 1656967. DOI: 10.1197/jamia.M2087. View

Shariff S, Cuerden M, Haynes R, McKibbon K, Wilczynski N, Iansavichus A . Evaluating the impact of MEDLINE filters on evidence retrieval: study protocol. Implement Sci. 2010; 5:58. PMC: 2917395. DOI: 10.1186/1748-5908-5-58. View

Schilling L, Steiner J, Lundahl K, Anderson R . Residents' patient-specific clinical questions: opportunities for evidence-based learning. Acad Med. 2004; 80(1):51-6. DOI: 10.1097/00001888-200501000-00013. View

Hoogendam A, Stalenhoef A, de Vries Robbe P, Overbeke A . Analysis of queries sent to PubMed at the point of care: observation of search behaviour in a medical teaching hospital. BMC Med Inform Decis Mak. 2008; 8:42. PMC: 2567311. DOI: 10.1186/1472-6947-8-42. View

10.

Delaney A, Bagshaw S, Ferland A, Laupland K, Manns B, Doig C . The quality of reports of critical care meta-analyses in the Cochrane Database of Systematic Reviews: an independent appraisal. Crit Care Med. 2007; 35(2):589-94. DOI: 10.1097/01.CCM.0000253394.15628.FD. View

11.

Hoogendam A, de Vries Robbe P, Stalenhoef A, Overbeke A . Evaluation of PubMed filters used for evidence-based searching: validation using relative recall. J Med Libr Assoc. 2009; 97(3):186-93. PMC: 2706446. DOI: 10.3163/1536-5050.97.3.007. View

12.

Moher D, Tetzlaff J, Tricco A, Sampson M, Altman D . Epidemiology and reporting characteristics of systematic reviews. PLoS Med. 2007; 4(3):e78. PMC: 1831728. DOI: 10.1371/journal.pmed.0040078. View

13.

Haynes R, McKibbon K, Wilczynski N, Walter S, Werre S . Optimal search strategies for retrieving scientifically strong studies of treatment from Medline: analytical survey. BMJ. 2005; 330(7501):1179. PMC: 558012. DOI: 10.1136/bmj.38446.498542.8F. View

14.

Herskovic J, Tanaka L, Hersh W, Bernstam E . A day in the life of PubMed: analysis of a typical day's query log. J Am Med Inform Assoc. 2007; 14(2):212-20. PMC: 2213463. DOI: 10.1197/jamia.M2191. View

15.

Hersh W, Crabtree M, Hickam D, Sacherek L, Friedman C, Tidmarsh P . Factors associated with success in searching MEDLINE and applying evidence to answer clinical questions. J Am Med Inform Assoc. 2002; 9(3):283-93. PMC: 344588. DOI: 10.1197/jamia.m0996. View

16.

Patel M, Schardt C, Sanders L, Keitz S . Randomized trial for answers to clinical questions: evaluating a pre-appraised versus a MEDLINE search protocol. J Med Libr Assoc. 2006; 94(4):382-7. PMC: 1629432. View

17.

Schardt C, Adams M, Owens T, Keitz S, Fontelo P . Utilization of the PICO framework to improve searching PubMed for clinical questions. BMC Med Inform Decis Mak. 2007; 7:16. PMC: 1904193. DOI: 10.1186/1472-6947-7-16. View

18.

Haynes R . Of studies, syntheses, synopses, summaries, and systems: the "5S" evolution of information services for evidence-based healthcare decisions. Evid Based Med. 2007; 11(6):162-4. DOI: 10.1136/ebm.11.6.162-a. View

19.

Lin J, Wilbur W . PubMed related articles: a probabilistic topic-based model for content similarity. BMC Bioinformatics. 2007; 8:423. PMC: 2212667. DOI: 10.1186/1471-2105-8-423. View

20.

Gonzalez-Gonzalez A, Dawes M, Sanchez-Mateos J, Riesgo-Fuertes R, Escortell-Mayor E, Sanz-Cuesta T . Information needs and information-seeking behavior of primary care physicians. Ann Fam Med. 2007; 5(4):345-52. PMC: 1934982. DOI: 10.1370/afm.681. View