Building a Performance Measurement Framework for Telephone Triage Services in Finland: a Consensus-making Study Based on Nominal Group Technique

Overview

Journal Scand J Trauma Resusc Emerg Med

Publisher Biomed Central

Specialty Emergency Medicine

Date 2024 Aug 13

PMID 39138499

Authors

Hanna Vainio

Leena Soininen

Paulus Torkki

Affiliations

Soon will be listed here.

Abstract

Background: There has been a significant expansion in the measurement of healthcare system performance. However, there is a lack of a comprehensive performance measurement framework to assess the effects of telephone triage services on the urgent care system. The aim of our Delphi study was to construct and validate a performance measurement framework designed explicitly for telephone triage services.

Methods: This study was conducted in Finland with a group of eight experienced senior physicians from the country's 20 largest joint emergency departments, serving over 90% of the population for urgent care. The Nominal Group Technique (NGT) was utilised to achieve consensus on measuring telephone triage performance. Initially, performance indicators (PIs) were identified through Delphi method rounds from December 10th to December 27th, 2021, with eight experts participating, and from December 29th, 2021, to January 23rd, 2022, where five of these experts responded. NGT further deepened these themes and perspectives, aiding in the development of a comprehensive performance measurement framework. The final framework validation began with an initial round from February 13th to March 3rd, 2022, receiving five responses. Due to the limited number of responses, an additional validation round was conducted from October 29th to November 7th, 2023, resulting in two more responses, increasing the total number of respondents in the validation phase to seven.

Results: The study identified a strong desire among professionals to implement a uniform framework for measuring telephone triage performance. The finalised framework evaluates telephone triage across five dimensions: service accessibility, patient experience, quality and safety, process outcome, and cost per case. Eight specific PIs were established, including call response metrics, service utility, follow-up care type and distribution, ICPC-2 classified encounter reasons, patient compliance with follow-up care, medical history review during assessment, and service cost per call.

Conclusions: This study validated a performance measurement framework for telephone triage services, utilising existing literature and the NGT method. The framework includes five key dimensions: patient experience, quality and safety, outcome of the telephone triage process, cost per case, and eight PIs. It offers a structured and comprehensive approach to measuring the overall performance of telephone triage services, enhancing our ability to evaluate these services effectively.

References

Justice J, Jang R . Tapping employee insights with the Nominal Group Technique. Am Pharm. 1990; NS30(10):43-5. DOI: 10.1016/s0160-3450(16)33610-8. View

Graversen D, Christensen M, Pedersen A, Carlsen A, Bro F, Christensen H . Safety, efficiency and health-related quality of telephone triage conducted by general practitioners, nurses, or physicians in out-of-hours primary care: a quasi-experimental study using the Assessment of Quality in Telephone Triage (AQTT) to assess.... BMC Fam Pract. 2020; 21(1):84. PMC: 7211335. DOI: 10.1186/s12875-020-01122-z. View

Lake R, Georgiou A, Li J, Li L, Byrne M, Robinson M . The quality, safety and governance of telephone triage and advice services - an overview of evidence from systematic reviews. BMC Health Serv Res. 2017; 17(1):614. PMC: 5577663. DOI: 10.1186/s12913-017-2564-x. View

Sun B, Hsia R, Weiss R, Zingmond D, Liang L, Han W . Effect of emergency department crowding on outcomes of admitted patients. Ann Emerg Med. 2012; 61(6):605-611.e6. PMC: 3690784. DOI: 10.1016/j.annemergmed.2012.10.026. View

Warren F, Calitri R, Fletcher E, Varley A, Holt T, Lattimer V . Exploring demographic and lifestyle associations with patient experience following telephone triage by a primary care doctor or nurse: secondary analyses from a cluster randomised controlled trial. BMJ Qual Saf. 2015; 24(9):572-82. PMC: 4552919. DOI: 10.1136/bmjqs-2015-003937. View

Fournier J, Escourrou B, Dupouy J, Bismuth M, Birebent J, Simmons R . Identifying competencies required for medication prescribing for general practice residents: a nominal group technique study. BMC Fam Pract. 2014; 15:139. PMC: 4129426. DOI: 10.1186/1471-2296-15-139. View

Graversen D, Pedersen A, Carlsen A, Bro F, Huibers L, Christensen M . Quality of out-of-hours telephone triage by general practitioners and nurses: development and testing of the AQTT - an assessment tool measuring communication, patient safety and efficiency. Scand J Prim Health Care. 2019; 37(1):18-29. PMC: 6454404. DOI: 10.1080/02813432.2019.1568712. View

Marshall S, Harrison J, Flanagan B . The teaching of a structured tool improves the clarity and content of interprofessional clinical communication. Qual Saf Health Care. 2009; 18(2):137-40. DOI: 10.1136/qshc.2007.025247. View

Porter M . What is value in health care?. N Engl J Med. 2010; 363(26):2477-81. DOI: 10.1056/NEJMp1011024. View

10.

McMillan S, King M, Tully M . How to use the nominal group and Delphi techniques. Int J Clin Pharm. 2016; 38(3):655-62. PMC: 4909789. DOI: 10.1007/s11096-016-0257-x. View

11.

Sullivan G . A primer on the validity of assessment instruments. J Grad Med Educ. 2012; 3(2):119-20. PMC: 3184912. DOI: 10.4300/JGME-D-11-00075.1. View

12.

Sexton V, Dale J, Atherton H . An evaluation of service user experience, clinical outcomes and service use associated with urgent care services that utilise telephone-based digital triage: a systematic review protocol. Syst Rev. 2021; 10(1):25. PMC: 7805218. DOI: 10.1186/s13643-021-01576-x. View

13.

Gibson A, Randall D, Tran D, Byrne M, Lawler A, Havard A . Emergency Department Attendance after Telephone Triage: A Population-Based Data Linkage Study. Health Serv Res. 2017; 53(2):1137-1162. PMC: 5867179. DOI: 10.1111/1475-6773.12692. View

14.

Pourmohammadi K, Hatam N, Shojaei P, Bastani P . A comprehensive map of the evidence on the performance evaluation indicators of public hospitals: a scoping study and best fit framework synthesis. Cost Eff Resour Alloc. 2018; 16:64. PMC: 6282321. DOI: 10.1186/s12962-018-0166-z. View

15.

Gamst-Jensen H, Christensen E, Lippert F, Folke F, Egerod I, Huibers L . Self-rated worry is associated with hospital admission in out-of-hours telephone triage - a prospective cohort study. Scand J Trauma Resusc Emerg Med. 2020; 28(1):53. PMC: 7288501. DOI: 10.1186/s13049-020-00743-8. View

16.

McMillan S, Sav A, Kelly F, King M, Whitty J, Wheeler A . Is the pharmacy profession innovative enough?: meeting the needs of Australian residents with chronic conditions and their carers using the nominal group technique. BMC Health Serv Res. 2014; 14:476. PMC: 4282516. DOI: 10.1186/1472-6963-14-476. View

17.

Morgan J, Muskett T . Interactional misalignment in the UK NHS 111 healthcare telephone triage service. Int J Med Inform. 2019; 134:104030. DOI: 10.1016/j.ijmedinf.2019.104030. View

18.

Knowles E, OCathain A, Turner J, Nicholl J . Effect of a national urgent care telephone triage service on population perceptions of urgent care provision: controlled before and after study. BMJ Open. 2016; 6(10):e011846. PMC: 5073559. DOI: 10.1136/bmjopen-2016-011846. View

19.

Kumar A, Ozdamar L, Ng C . Procurement performance measurement system in the health care industry. Int J Health Care Qual Assur Inc Leadersh Health Serv. 2005; 18(2-3):152-66. DOI: 10.1108/09526860510588179. View

20.

Vilstrup E, Graversen D, Huibers L, Christensen M, Pedersen A . Communicative characteristics of general practitioner-led and nurse-led telephone triage at two Danish out-of-hours services: an observational study of 200 recorded calls. BMJ Open. 2019; 9(6):e028434. PMC: 6596995. DOI: 10.1136/bmjopen-2018-028434. View