Implementation costs of telephone nurse triage service

Custos de implantação do serviço de triagem de enfermagem por telefone

Costos de implantación del servicio de triaje telefónico de enfermería

Abstracts

Telephone nurse triage services are becoming increasingly common in healthcare systems worldwide. Florianópolis (Santa Catarina State, Brazil) is the first municipality in the country to provide this service in its public health system. This study adopted a quantitative, descriptive, and analytical methodology to evaluate the impact of this program on overall costs of the public health system. The research examined all 33,869 calls received by the telephone triage service from March 16 to October 31 in 2020, and calculated the program costs during the period. Avoided cost were calculated by the difference between estimated consultation costs considering patient-stated first alternative and the program recommendation after triage. Analyzing only the costs for the municipality of Florianópolis, the program’s costs exceeded avoided costs by almost BRL 2.5 million during the period. By expanding the analysis to include costs of emergency department consultation - not administered by the municipality - based on data from previous research, we found that the program spares BRL 34.59 per call, a 21% cost reduction for the health system. Considering the preliminary results of the study and its limitations, it is understood that the service of telephone nurse triage can reduce costs in the healthcare system.

Keywords:
Primary Health Care; Telemedicine; Costs and Cost Analysis; Public Health Systems; Emergency Medical Services

Vários países oferecem serviços de triagem de enfermagem por telefone em seus sistemas de saúde. Florianópolis (Santa Catarina, Brasil) é o primeiro município do país a prestar esse serviço no sistema público de saúde. Este estudo adotou uma metodologia quantitativa, descritiva e analítica para avaliar se a implementação do programa reduziu os custos gerais do sistema público de saúde. A pesquisa examinou todas as 33.869 ligações submetidas à triagem telefônica entre 16 de março e 31 de outubro de 2020 e os custos do programa no período. Os custos evitados foram calculados pela diferença entre os custos estimados da consulta, considerando a primeira alternativa declarada pelo paciente e a recomendação do programa após a triagem. Analisando apenas os custos para o Município de Florianópolis, os resultados mostram que os custos do programa superaram os custos evitados em quase BRL 2,5 milhões no período. Ampliando a análise que engloba os custos da consulta ao pronto-socorro, os quais não são administrados pelo Município de Florianópolis, e com base em dados de pesquisas anteriores, os resultados mostram que o programa gera uma economia de BRL 34,59 por chamada, uma redução de custos de 21% para o sistema de saúde. Considerando os resultados preliminares do estudo, bem como as limitações apontadas, entende-se que o serviço de triagem de enfermagem por telefone pode contribuir para a redução de custos no sistema de saúde.

Palavras-chave:
Atenção Primária à Saúde; Telemedicina; Custos e Análise de Custo; Sistemas Públicos de Saúde; Serviços Médicos de Emergência

Varios países cuentan con servicios de triaje telefónico de enfermería en sus sistemas de salud. Florianópolis (Santa Catarina, Brasil) es el primer municipio del país en ofertar este tipo de servicio en el sistema de salud pública. Este estudio utilizó una metodología cuantitativa, descriptiva y analítica para evaluar si la implementación del programa reduce los costos generales del sistema de salud público. La encuesta examinó las 33.869 llamadas enviadas al triaje telefónico entre el 16 de marzo y el 31 de octubre de 2020, y los costos del programa durante ese periodo. Los costos evitados se calcularon por la diferencia entre los costos estimados de la consulta, considerando la primera alternativa informada por el paciente y la recomendación del programa después del triaje. El análisis solo de los costos para el municipio de Florianópolis apunta que los costos del programa sobrepasaron los costos evitados en casi BRL 2,5 millones en el periodo. En el análisis más amplio que abarca los costos de consulta y de urgencias, que no son administrados por el municipio de Florianópolis, y con base en datos de investigaciones anteriores, los resultados muestran que el programa genera ahorros de BRL 34,59 por llamada, una reducción del 21% en los costos para el sistema de salud. Los resultados preliminares del estudio y las limitaciones señaladas constatan que el servicio de triage telefónico de enfermería puede contribuir a la reducción de costos en el sistema de salud.

Palabras-clave:
Atención Primaria de Salud; Telemedicina; Costos y Análisis de Costo; Sistemas Públicos de Salud; Servicios Médicos de Urgencia

Introduction

Accessing health services is difficult and the high costs of care in urgent care centers are an issue in many countries. Telephone nurse triage services have been implemented in national health systems worldwide to provide out-of-hours primary health care. Countries such as the United Kingdom, the Netherlands, and Denmark have been using these services since the 1990s, whereas other European countries, Canada, the United States, and Australia have adopted them more recently ^1,2,3,4,5,6.

The service consists of a call center operated by physicians and/or nurses, which the patient must call before seeking face-to-face health services. Professionals provide the first consultation via telephone, using electronic systems with algorithms and protocols designed for each clinical situation. These professionals recommend services such as consultation in an urgent or health care center, going to an emergency department (ED), scheduling a next-day appointment with the family health strategy physician, calling an ambulance, offering remote consultation, or self-care guidance when medical attention is unnecessary ^1,2,3,4,5,6.

Inspired by successful international experiences, the municipality of Florianópolis (Santa Catarina State) implemented the Brazil’s first public telephone triage service, Alô Saúde Floripa (ASF; Hello Health Floripa), on March 16, 2020. The service consists of a 24-hour call center, free-of-charge, with nurses and nursing technicians who attend patients seeking medical care or guidance.

ASF was introduced based on studies on telephone triage services in other countries ^1,7,8,9, aiming at reducing the number of people with minor health problems unnecessarily using emergency services in local health units. The program aims to guide patients to the most appropriate service in each case, improving healthcare quality and reducing costs.

The literature has shown that these services can reduce costs for the health system ^10,11, are safe, efficient, and of good quality ^4,12,13. Studies demonstrate that patients tend to follow the recommendations received ^{7,9,14,15,16,17} and that telephone triage services are responsive, efficient ^5,8,18, and present a high level of patient satisfaction ^1,19,20.

Since ASF is a pioneer experience, there are no studies in Brazil regarding the results and costs of this service. Thus, this study is the first effort to evaluate whether ASF has achieved its goal of reducing the public health system overall costs.

Methodology

This study assessed all calls made to ASF from March 16 to October 31, 2020. The study examined the costs of implementing and maintaining the program and the cost of all services provided at the municipality’s primary health care centers (PHCC) and urgent care centers (UCC) during the same period. This is a study on the Florianópolis health system. The telephone nurse triage service is being analyzed in comparison to the current practice of patients seeking health units in the Florianópolis region without prior guidance and by spontaneous demand. The Monitoring Committee of Health Research Projects of the Florianópolis Health Department (SMS) approved this research (approval document: OE21/SMS/GAB/ESP/2021).

This is a quantitative, descriptive, and analytical research based on data from records of calls made to ASF and from the Health Cost Management System of the SMS. Additionally, document analysis was conducted to examine the contract signed between the SMS and the service provider, TopMed (https://topmed.com.br/). Quantitative research was considered the most appropriate approach since cost analysis requires objective and measurable data, which was already organized and systematized. The information on ASF services was collected from the electronic medical record system used by the hired company. These records are available for both program monitoring by the SMS and for research purposes.

The telephone nurse triage service in Florianópolis consists of a call center with several workstations equipped with telephone sets and computers with internet access. The call center is operated by nurses and nursing technicians who handle patients’ inquiries and categorize them into two complexity levels. At Level 1, nursing technicians attend patients without any clinical complaints who need support with administrative or bureaucratic issues. At Level 2, nurses address patients with health problems who seek medical attention or guidance. The nurses refer patients to the most appropriate service, via a computerized system with algorithms and protocols designed to address the complaint based on the patients’ answers to structured questionnaires. For all calls directed to Level 2 (symptomatic cases), a follow-up call is made 24 hours later to assess the patient’s condition, verifying whether the patient followed the instructions provided, sought the recommended health services, had a consultation, and determine their level of satisfaction. In addition, patients are asked what they would have done if ASF had not been available (patient-stated first alternative): going to an ED, a UCC, a PHCC, calling an ambulance, or staying at home. Box 1 shows the source of information, collection procedure, and information gap for each dimension.

Box 1
Dimensions of the analysis model and their information source, collection procedure, and information gap.

DIMENSION	INFORMATION SOURCE	COLLECTION PROCEDURE	INFORMATION GAP
Direct costs per consultation	Annual report of the Health Cost Management System of the Florianópolis Health Department	Document analysis	Cost of medical consultations in health units (PHCC and UCC)
ASF program costs of implementation and maintenance	The contract between the municipality and the company that operates ASF	Document analysis	ASF implementation and maintenance costs
Avoided costs	ASF business intelligence report	Document analysis	Avoided costs regarding medical consultations, obtained by comparing the patient-stated first alternative and the ASF recommendation

• ASF: Alô Saúde Floripa; PHCC: primary health care centers; UCC: urgent care centers.
• Source: prepared by the authors.

The direct costs per consultation were calculated by considering: (1) personnel cost; (2) general expenses, comprising: rent, water, electricity, telephone, internet, electronic medical record system, printing, cleaning, security, vehicles, equipment maintenance, building maintenance, construction, laundry, and others; (3) consumption of materials and supplies such as oxygen, hygiene, materials for community health agents, nursing materials and equipment, medical materials and equipment, acupuncture supplies, personal protective equipment, foodstuffs, clothing, uniforms, printed material, judicial costs, and others. The costs were distributed via screening and partitioning. Screening was used for items purchased for a specific health unit, whereas partitioning was used for items distributed throughout various health units in the local health system, and it was not possible to attribute the cost to only one unit. In the latter case, the apportionment was based on the health units size.

The health units absorbed the costs of personnel, general expenses (including physical structure and services), and supplies and materials. The costs were identified through receipts, invoices, or accounts related to direct consumption or, in a few cases, partitioning. The absorption costing method was used to calculate the product unit costs, including direct materials, direct labor, and indirect production costs, both variable and fixed. This method allows for an overview of the costs of the health service system based on cost centers. Each cost center absorbs costs for personnel, consumables, third-party services, and general expenses.

The costs of personnel were collected from the workers’ payroll and allocated directly to their health unit. This costs included total salaries, overtime, on-call hours, on-call notice, and employer contribution to the employees’ social security. The research calculated the costs of personnel considering all professions per cost center, except oral health professionals, as they are not involved in medical or nursing consultations.

Data collected included the facilities structure, human resources, and production of each health unit. These costs were analyzed to verify the cost per medical consultation in the health and urgent care centers. Additionally, the contract signed between the municipality of Florianópolis and the TopMed company was examined. The company won the bid in a public procurement process launched to hire an operator for the program, and the contract examination aimed to verify the monthly cost of services to the SMS. The contract was available in the municipality’s official gazette.

Firstly, we calculated the consultation cost based on patient-stated first alternatives if ASF had not been available during the research period. Then, we also calculated consultation cost based on the ASF recommendation to patients, adjusted by the percentage of patient adherence to the recommendations, as reported in the follow-up phone call. Finally, consultation cost according to the patient-stated first alternative was subtracted by consultation cost based on the recommendations corrected by adherence, resulting in the avoided costs.

The calculation of consultation costs at different types of health units in the municipal health system was conducted for UCC and PHCC. For the UCC, medical consultations were considered as the product in the calculation, since nurses perform case screening and risk classification, establishing urgency and priorities, but they do not carry out consultations, prescribe medication, or request tests. As for the PHCC, both medical and nursing consultations were considered in the calculation of consultation costs since nurses conduct consultations, prescribe medication, and request exams, based on the nursing protocols prepared by the SMS, with physicians’ support through inter-consultations when necessary.

The costs of implementing and maintaining ASF were obtained by analyzing contract number 1116/FMS/2019 signed between the SMS and TopMed. Implementation costs were established based on the amount paid to the company from the contract’s signing on February 12, 2019, until the start of operation on March 16, 2020. During this period, the company’s algorithms were revised and adapted to the needs of the SMS. Moreover, the company hired and trained employees and adapted their infrastructure. The maintenance cost was defined as the amount paid to the company from March 16, 2020, to October 31, 2020.

In the analyses of patients-stated first alternatives and ASF recommendations, UCC and ED were grouped as urgent care center since both services have similar characteristics of care and patients may refer to them interchangeably. It was not possible to access the costs of care at EDs since they are managed by the state government. Similarly, the first alternatives and recommendations stated for PHCC, elective consultation, COVID-19 cases in PHCCs, PHCC within 24h, scheduled consultation at the PHCC within 12h, and suspected cases to be informed to epidemiological surveillance were grouped as PHCC since all these services direct patients to consultations at this type of health unit.

Results

The average direct costs per consultation according to the health unit type in the first half of 2020 in the local health system in Florianópolis were BRL 168.52 for UCCs and BRL 140.73 for PHCC.

The total cost of implementing and maintaining ASF during the study period was BRL 4,174,500.00, with BRL 2,846,250.00 allocated to maintenance and BRL 1,328,250.00 to implementation. However, the implementation cost may be overestimated as it is a one-time cost that could be diluted over the entire service period, which can extend over several years since the contract is renewable and would only require maintenance costs.

Table 1 presents the estimated consultation costs per type of health unit during the study period according to the patient-stated first alternative if ASF had not been available. It also shows the estimated costs of services per type of health unit according to the program recommendation.

According to the patients’ answers in follow-up calls, adherence to ASF recommendations was 95.68%. Thus, estimated costs must be adjusted to account for non-adherence. Following, we show the amounts adjusted by 4.32%: estimated cost of recommendation (BRL 2,185,853.93); consultation cost for non-adhering patients (BRL 94,428.89); and estimated cost of recommendation - adjusted (BRL 2,280,282.82).

Table 1
Estimated consultation costs from March 16, 2020, to October 31, 2020, per type of healthcare unit according to the patient-stated first alternative and per healthcare unit considering the recommendations from Alô Saúde Floripa (ASF). Florianópolis, Santa Catarina State, Brazil.

Type of healthcare unit	Consultations	Cost per consultation (BRL)	Total cost (BRL)
According to the patient-stated first alternative
UCC	16,733	168.52	2,819,868.43
PHCC	8,385	140.73	1,180.061.87
Total			3,999,930.30
Considering the recommendations from ASF
UCC	2,688	168.52	452,985.50
PHCC	12,313	140.73	1,732,868.43
Total			2,185,853.93

• PHCC: primary health care centers; UCC: urgent care centers.
• Source: prepared by the authors.

During the study period, the estimated avoided costs due to the program activities were: estimated cost for patient-stated first alternative (BRL 3,999,930.30); estimated costs for recommendation - adjusted (BRL 2,280,282.82); and avoided costs (BRL 1,719,647.48).

The average cost per consultation based on the patient-stated first alternative per patient was BRL 155.49. The average cost per consultation based on the program recommendation, adjusted for adherence rate, was BRL 88.64. The difference between these values was BRL 66.85 per patient, representing a 57% saving per patient.

The difference between the avoided cost (BRL 1,719,647.48) and the total cost of ASF (BRL 4,174,500.00) from March 16, 2020, to October 31, 2020, was BRL 2,454,852.52.

This study did not include EDs as they are state-managed health units, and data on the consultation costs were not available. However, since EDs in the municipality operate 24 hours a day and provide out-of-hours care, the cost of ED consultation was estimated based on previous studies, and the avoided cost was then calculated. Table 2 shows the cost of consultations in urgent care services - equivalent to the Brazilian UCCs analyzed in this study -, PHCCs, and EDs in previous international studies, demonstrating the proportion between these costs.

Table 2
Consultation cost per type of health service and the ratio between these costs, according to previous studies.

Study (Year/Country)	PHCC (cost)	UCC (cost)	ED (cost)	ED/UCC	ED/ PHCC	UCC/ PHCC
Marklund et al. 10 (2007/Sweden)	EUR 100	*	EUR 250	*	2.5	*
O’Connel et al. 21 (2001/United States)	USD 80	USD 95	USD 309	3.25	3.86	1.18
Cariello 26 (2003/United States)	USD 82	USD 135	USD 269	1.99	3.28	1.64
Lambert et al. 24 (2013/United Kingdom)	GBP 19	GBP 28	GBP 111	3.96	5.84	1.47

• ED: ermergency department; PHCC: primary health care centers; UCC: urgent care centers.
• Source: prepared by the authors.
• * No data available.

Based on these data, a similar proportion can be observed between consultation costs in the studies analyzed. The average ratio between the cost of an urgent care center consultation and a PHCC consultation was 1.43, and the ratio between the cost of an ED consultation and a PHCC consultation in our study was 1.19, which is consistent with previous studies. The average ratio between the cost of an ED consultation and the cost of a UCC consultation in previous studies was 3.06. Applying this ratio to the cost of a UCC consultation in our study (BRL 168.52) presented an estimated cost of BRL 515.67 for an ED consultation. Applying the average ratio between the cost of an ED consultation and the cost of a PHCC consultation in previous studies (3.87) to the cost of a PHCC consultation in this study (BRL 140.73) presented an estimated cost of BRL 544.62 for an ED consultation. The mean of these two values was BRL 530.14 for the estimated cost of an ED consultation.

The costs avoided during the study period may be reached based on the data analysis on patient-stated first alternative and ASF recommendation, and separately calculating the costs of UCC consultation and an ED consultation. The adjusted cost (considering the patients’ adherence to recommendations) was BRL 5,982,058.18, as shown in Table 3. After discounting ASF operating costs for the period, the total savings for the public health system of the municipality amounted to BRL 1,807,558.18, which represents savings of BRL 34.59 per call or a 21% cost reduction.

Table 3
Consultation cost per patient-stated first alternative and recommendations adjusted for non-adherence and avoided costs.

	Consultations	Cost per consultation (BRL)	Total cost (BRL)
Cost of patient-stated first alternative
UCC	4,005	168.52	674,928.17
ED	12,728	530.14	6,747,621.92
PHCC	8,385	140.73	1,180,061.87
Total			8,602,611.96
Cost of program recommendation
UCC	1,786	168.52	300,979.20
ED	902	530.14	478,186.28
PHCC	12,313	140.73	1,732,868.43
Total			2,512,033.91
Non-adhesion		4.32%	108,519.86
Adjusted			2,620,553.78
Avoided costs adjusted			5,982,058.18

• ED: ermergency department; PHCC: primary health care centers; UCC: urgent care centers.
• Source: prepared by the authors.

Discussion

The results show that ASF fulfilled its goal of directing patients - based on the symptoms assessed during the phone call - to less complex care than they had envisioned. Also, the patients declared to adhere to the program recommendations most of the time. Despite the avoided costs summing BRL 1,719,647.48 for the local health system during the study period, the program total cost for the municipality of Florianópolis exceeded this amount by BRL 2,454,852.52. However, when considering the estimated cost of an ED consultation, which is managed by the state government in Brazil, we noticed a cost reduction of BRL 1,807,558.18 in the period, or BRL 34.59 per call made to the program - i.e., a cost reduction of 21%.

In other countries where similar services have been implemented, the entire health system is managed by one organization, including PHCCs, UCCs, EDs, ambulance services, and telephone triage services. Thus, savings are more evident, mainly in reducing ED consultations, which are redirected to other places of less complex care or avoided through self-care guidelines.

These findings reveal a problematic aspect of the Brazilian health system’s organization, where each state and municipality are responsible for the administration and financing of different services. The municipalities manage primary health care, whereas the state or federal governments manages most hospital network. This configuration presents challenges to implementing integrated programs, as the consequences of activities conducted by one level may bring benefits to only other level, reducing incentives. For example, the municipalities’ investment in primary health care may reduce hospitalizations in the state’s hospital network due to primary health care sensitive conditions. In this case, the city government made all investment , while the state was benefited the most through a cost reduction. Thus, the local government has little incentive to keep investing. This example resembles what happens in the case of ASF.

The cost reduction of the program may be greater in the coming years since the implementation cost is a one-time expense that was only present in the first year of the contract. Since this is the first experience of such program in Brazil and the cost per call is higher than in other studies, it may be overrated, and future costs could be reduced if more companies offer similar services at a lower price. This would also increase the program cost reduction for the health system.

However, the research has some limitations related to the quantitative nature of the analyses and the difficulty in assessing health costs. The proposed model could not measure other benefits from directing patients to the most appropriate healthcare modality, be it at home, at a PHCC, UCC, or ED. These benefits include patient safety, avoiding unnecessary and potentially harmful examinations and treatments, avoiding time waste and unnecessary travel, decreasing social costs, and ensuring longitudinal care for chronic health problems of patients directed to primary health care.

The study was also unable to determine whether the number of consultations at ED, UCC, and PHCC in the municipality of Florianópolis reduced during the studied period due to the program activities. The COVID-19 pandemic, which occurred in the same months, significantly influenced the entire system, jeopardizing the correlation between the factors. In addition, data on costs of ED consultations in the municipality were not available since the state government manages these facilities, so we had to estimate these costs.

The ASF program can be considered an innovation in Brazil, supported by successful experiences worldwide and potential benefits to the Brazilian public health system. Therefore, this study offers valuable theoretical contributions by reviewing international literature and analyzing the program first experience in Brazil. The study provides the program profile and examines their costs and impact on the total costs of the local health system.

In 2001, a study in the United States evaluated the return on investment for a telephone nurse triage service ²¹. The authors estimated the operational costs in one year, including the implementation costs, at USD 382,636. Total implementation cost was USD 12,500 over three years, resulting in an annual cost of USD 4,167. Another U.S. study, published in 2014 ²², estimated the cost of a telephone nurse triage service at USD 8.7 million in one year. The program served a population of 3.1 million people and received 63,277 calls in one year.

A Swedish study ¹⁰ calculated the cost reduction per call in a telephone triage service by adding the amount of avoided costs and dividing the result by the number of phone calls received, reaching EUR 77 in savings per call. In the United States ²³, a study that interviewed 266 patients who used a telephone triage service over one month. These patients were interviewed, and the study compared the costs of the patient-stated first alternative with the actual post-call actions. The research found savings of USD 14,568, which represents savings of USD 54 per call. A similar study conducted in the United Kingdom showed a difference of GBP 19 per patient and savings of 36% per patient ²⁴.

The evaluation report on the implementation of NHS 111 in four locations in the United Kingdom ²⁵ found conflicting values, with the service decreasing costs in some locations while increasing them in others. The analysis of the four locations together showed an increase of GBP 12.21 per call in the total cost of the health system. For the authors, these findings ​​should be cautiously interpreted due to differences in the urgency and emergency services of the four locations and variations in the NHS 111 operation in those places.

A U.S. study that evaluated more than 10,000 calls to the local telephone triage service ²¹ found a total saving of USD 775,853, and the cost of the service in the period was USD 382,636, i.e., a return on investment of USD 1.70 for each dollar invested in the program. It was not possible to estimate the amount saved per call because the total number of calls was not disclosed in the study. Another U.S. study assessed savings based on the patient-stated first alternative and what they actually did after calling the triage service ²⁶, finding savings of USD 54.42 per call.

Future studies should assess the total costs for the health system as a whole, not only for the local health system, and should include other levels of government, conducting an analysis based on estimates offered in previous studies. This research was limited to the data available from the SMS. An analysis of the impact of similar services on the number of consultations at different points of the health system would also be important to assess whether this service actually reduces consultations in ED, UCC, and PHCC. In addition, cross-checking data analysis could contribute to evaluating whether patients who called the service sought or did not seek care at the place they were referred to. This information may be collected from attendance records, analyzing whether the use of the health system by these patients was greater in the days following the call compared with the use of the system by patients who did not call the service.

Conclusion

This study highlights the potential benefits of implementing telephone nurse triage service to handle out-of-hours medical consultations in the Brazilian public health system. We demonstrated that this service can reduce total healthcare costs. However, the analysis of healthcare costs is still incipient in the country, mainly in public health and, especially, in primary healthcare. Therefore, further analyses of this type are essential for decision-making at all levels of government, considering their strong economic factor. Further studies focused on the analysis of healthcare services and interventions - also from the economic viewpoint - are crucial to promote better public health resources.

References

• ¹
  Bunn F, Byrne G, Kendall S. Telephone consultation and triage: effects on health care use and patient satisfaction. Cochrane Database Syst Rev 2004; (4):CD004180.
• ²
  Moll van Charante E, Giesen P, Mokkink H, Oort F, Grol R, Klazinga N, et al. Patient satisfaction with large-scale out-of-hours primary health care in The Netherlands: development of a postal questionnaire. Fam Pract 2006; 23:437-43.
• ³
  Buja A, Toffanin R, Rigon S, Sandona P, Carraro D, Damiani G, et al. Out-of-hours primary care services: demands and patient referral patterns in a Veneto region (Italy) Local Health Authority. Health Policy 2015; 119:437-46.
• ⁴
  Lake R, Georgiou A, Li J, Li L, Byrne M, Robinson M, et al. The quality, safety and governance of telephone triage and advice services - an overview of evidence from systematic reviews. BMC Health Serv Res 2017; 17:614.
• ⁵
  Posocco A, Scapinello MP, De Ronch I, Castrogiovanni F, Lollo G, Sergi G, et al. Role of out of hours primary care service in limiting inappropriate access to emergency department. Intern Emerg Med 2018; 13:549-55.
• ⁶
  Huibers L, Carlsen AH, Moth G, Christensen HC, Riddervold IS, Christensen MB. Patient motives for contacting out-of-hours care in Denmark: a cross-sectional study. BMC Emerg Med 2020; 20:20.
• ⁷
  Purc-Stephenson RJ, Thrasher C. Patient compliance with telephone triage recommendations: a meta-analytic review. Patient Educ Couns 2012; 87:135-42.
• ⁸
  Huibers L, Moth G, Carlsen AH, Christensen MB, Vedsted P. Telephone triage by GPs in out-of-hours primary care in Denmark: a prospective observational study of efficiency and relevance. Br J Gen Pract 2016; 66:e667-73.
• ⁹
  Tran DT, Gibson A, Randall D, Havard A, Byrne M, Robinson M, et al. Compliance with telephone triage advice among adults aged 45 years and older: an Australian data linkage study. BMC Health Serv Res 2017; 17:512.
• ¹⁰
  Marklund B, Strom M, Mansson J, Borgquist L, Baigi A, Fridlund B. Computer-supported telephone nurse triage: an evaluation of medical quality and costs. J Nurs Manag 2007; 15:180-7.
• ¹¹
  Langabeer 2nd JR, Champagne-Langabeer T, Alqusairi D, Kim J, Jackson A, Persse D, et al. Cost-benefit analysis of telehealth in pre-hospital care. J Telemed Telecare 2017; 23:747-51.
• ¹²
  Huibers L, Smits M, Renaud V, Giesen P, Wensing M. Safety of telephone triage in out-of-hours care: a systematic review. Scand J Prim Health Care 2011; 29:198-209.
• ¹³
  Graversen DS, Christensen MB, Pedersen AF, Carlsen AH, Bro F, Christensen HC, et al. Safety, efficiency and health-related quality of telephone triage conducted by general practitioners, nurses, or physicians in out-of-hours primary care: a quasi-experimental study using the Assessment of Quality in Telephone Triage (AQTT) to assess audio-recorded telephone calls. BMC Fam Pract 2020; 21:84.
• ¹⁴
  Nijland N, Cranen K, Boer H, van Gemert-Pijnen JE, Seydel ER. Patient use and compliance with medical advice delivered by a web-based triage system in primary care. J Telemed Telecare 2010; 16:8-11.
• ¹⁵
  Rimner T, Blozik E, Begley C, Grandchamp C, von Overbeck J. Patient adherence to recommendations after teleconsultation: survey of patients from a telemedicine centre in Switzerland. J Telemed Telecare 2011; 17:235-9.
• ¹⁶
  Williams B, Warren S, McKim R, Janzen W. Caller self-care decisions following teletriage advice. J Clin Nurs 2012; 21:1041-50.
• ¹⁷
  Gibson A, Randall D, Tran DT, Byrne M, Lawler A, Havard A, et al. Emergency department attendance after telephone triage: a population-based data linkage study. Health Serv Res 2018; 53:1137-62.
• ¹⁸
  Moth G, Huibers L, Christensen MB, Vedsted P. Out-of-hours primary care: a population-based study of the diagnostic scope of telephone contacts. Fam Pract 2016; 33:504-9.
• ¹⁹
  Kruse CS, Krowski N, Rodriguez B, Tran L, Vela J, Brooks M. Telehealth and patient satisfaction: a systematic review and narrative analysis. BMJ Open 2017; 7:e016242.
• ²⁰
  Zinger ND, Blomberg SN, Lippert F, Collatz Christensen H. Satisfaction of 30 402 callers to a medical helpline of the Emergency Medical Services Copenhagen: a retrospective cohort study. BMJ Open 2019; 9:e029801.
• ²¹
  O'Connell JM, Johnson DA, Stallmeyer J, Cokingtin D. A satisfaction and returna-on-investment study of a nurse triage service. Am J Manag Care 2001; 7:159-69.
• ²²
  Navratil-Strawn JL, Hawkins K, Wells TS, Ozminkowski RJ, Hawkins-Koch J, Chan H, et al. Listening to the nurse pays off: an integrated Nurse HealthLine programme was associated with significant cost savings. J Nurs Manag 2014; 22:837-47.
• ²³
  Bodgan GM, Green JL, Swanson D, Gabow P, Dart RC. Evaluating patient compliance with nurse advice line recommendations and the impact on healthcare costs. Am J Manag Care 2004; 10:534-42.
• ²⁴
  Lambert R, Fordham R, Large S, Gaffney B. A cost-minimisation study of 1.001 NHS Direct users. BMC Health Serv Res 2013; 13:300.
• ²⁵
  Turner J, O'Cathain A, Knowles E, Nicholl J, Tosh J, Sampson F, et al. Evaluation of NHS 111 pilot sites. Final report. Sheffield: Medical Care Research Unity, The University of Sheffield; 2012.
• ²⁶
  Cariello FP. Computerized telephone nurse triage: an evaluation of service quality and cost. J Ambul Care Manage 2003; 26:124-37.

Additional information

• Additional information

  ORCID: Ricardo Collar Rebolho (0000-0002-7797-8037); Fabiano Maury Raupp (0000-0001-9533-2574).

Publication Dates

• Publication in this collection
  14 Apr 2023
• Date of issue
  2023

History

• Received
  24 May 2022
• Reviewed
  31 Oct 2022
• Accepted
  25 Jan 2023