MOBILE APPLICATIONS FOR PATIENT SAFETY: A SCOPING REVIEW

APLICACIONES MÓVILES PARA LA SEGURIDAD DEL PACIENTE: REVISIÓN DE ALCANCE

ABSTRACT

Objective:

to map scientific production related to patient safety applications.

Method:

scoping review based on the method proposed by the JBI and the recommendations of the Preferred Reporting Items for Systematic Review and Meta-Analyses for Scoping Reviews. The search took place between February and March 2022 in six databases, gray literature and intellectual property registration offices. The review was conducted by peers.

Results:

a total of 28 studies were analyzed. The international goals most often covered by the software were the prevention of medication errors, safe surgery and efficient communication. The applications used different operating systems and mostly described the process of content selection and software architecture. The evaluation system used usability scales, before and after tests and clinical trials.

Conclusion:

the construction of healthcare applications, especially those aimed at implementing patient safety targets, is a broad field to be explored. It is necessary to strengthen the quality of the information offered and the involvement of a multi-professional team in its development. There are gaps in the methodological designs and a lack of legislation regulating aspects related to the quality and veracity of the information provided by the software. Research protocol registered in the Open Science Framework (https://osf.io/8b9pz/)

DESCRIPTORS:
Mobile applications; Patient safety; Perioperative nursing; Education in health; Perioperative care.; Medical Informatics Applications

RESUMEN

Objetivo:

mapear la producción científica relacionada con aplicaciones orientadas a la seguridad del paciente.

Método:

revisión de alcance basada en el método propuesto por JBI y en las recomendaciones previstas en Preferred Reporting Items for Systematic Review and Meta-Analyses for Scoping Reviews. La búsqueda se realizó entre los meses de febrero y marzo de 2022 en seis bases de datos, literatura gris y oficinas de registro de propiedad intelectual. Se realizó la revisión por pares.

Resultados:

se analizaron 28 estudios. Los objetivos internacionales más contemplados por el software fueron la prevención de errores de medicación, cirugía segura y comunicación eficiente. Las aplicaciones utilizaron diferentes sistemas operativos y en su mayoría describieron el proceso de selección de contenido y la arquitectura del software. El sistema de evaluación incluyó el uso de escalas de usabilidad, antes y después de pruebas y ensayos clínicos.

Conclusión:

la construcción de aplicaciones de salud, especialmente aquellas destinadas a implementar objetivos de seguridad del paciente, es un campo amplio por explorar. Es necesario reforzar la calidad de la información ofrecida e implicar a un equipo multidisciplinario para su desarrollo. Se advierten lagunas en cuanto a diseños metodológico, como también la falta de legislación que regule aspectos relacionados con la calidad y veracidad de la información que ofrece el software. Protocolo de investigación registrado en el Open Science Framework (https://osf.io/8b9pz/)

DESCRIPTORES:
Aplicaciones móviles; Seguridad del paciente; Enfermería perioperatoria; Educación en salud; Cuidados perioperatorios; Aplicaciones de la informática médica

RESUMO

Objetivo:

mapear a produção científica relacionada a aplicativos voltados para segurança do paciente.

Método:

revisão de escopo baseada no método proposto pelo JBI e nas recomendações do Preferred Reporting Items for Systematic Review and Meta-Analyses for Scoping Reviews. A busca ocorreu entre os meses de fevereiro e março de 2022 em seis bases de dados, literatura cinzenta e escritórios de registro de propriedade intelectual. A revisão foi realizada por pares.

Resultados:

foram analisados 28 estudos. As metas internacionais mais contempladas pelos softwares foram a prevenção de erros de medicação, cirurgia segura e comunicação eficiente. Os aplicativos usaram diferentes sistemas operacionais e descreveram em sua maioria o processo de seleção de conteúdo e arquitetura do software. O sistema de avaliação contou com uso de escalas de usabilidade, testes antes e depois e ensaios clínicos.

Conclusão:

a construção de aplicativos em saúde, em especial, os voltados para aplicação das metas de segurança do paciente é um amplo campo a ser explorado. Faz-se necessário fortalecer a qualidade das informações ofertadas e o envolvimento de uma equipe multiprofissional para seu desenvolvimento. Observam-se lacunas relativas aos desenhos metodológicos e carência quanto às legislações que regulamentam aspectos relacionados à qualidade e veracidade das informações ofertadas pelos softwares. Protocolo de pesquisa registrado no Open Science Framework (https://osf.io/8b9pz/)

DESCRITORES:
Aplicativos móveis; Segurança do paciente; Enfermagem perioperatória; Educação em saúde; Assistência perioperatória; Aplicações da informática médica

INTRODUCTION

Errors in care have always been a reality, but they reached greater visibility after the report "To err is human", published at the end of the 1990s, which identified that more people died in the United States from errors in care than from cancer and HIV1.

Based on this report, the World Health Organization (WHO) began to draw up strategies for patient safety, such as the creation of the global alliance for patient safety in 2004 and the launch of three global challenges: hand hygiene, safe surgeries and harm-free medication2.

In Brazil, in 2013, the National Patient Safety Program was established3, proposing six protocols, namely hand hygiene; safe surgery; fall prevention; safety in prescribing; use and administration of medication; prevention of pressure injuries and patient identification. In addition, the program provides for the promotion of research and the encouragement of technical and operational innovations that can mitigate the occurrence of errors3.

Patient safety today is understood as a set of activities that allow for the creation of a culture, structure, processes, conduct and technologies to make it possible to reduce the risks related to healthcare, as well as reducing their impact should they occur4.

Despite all the efforts involved in improving and promoting patient safety, it is still estimated that one in ten patients in high-income countries and one in four patients in low and middle-income countries are subject to preventable errors. As a result, in addition to an increase in health care costs, these errors contribute to around 2.6 million deaths4.

Technological innovation can help achieve patient safety goals, either by involving the patient themselves in this process, or by developing systems that directly assist the healthcare professional5^-6.

The use of technology to support healthcare is increasingly present in people's lives, with the use of m-Healths. Within this spectrum, the consumption of health apps that can be used for different purposes and with different functionalities stands out. Because they are always accessible, they allow patients to manage their pathologies, broaden their knowledge through educational resources and allow direct contact with health professionals7.

Thousands of apps are launched every year, and in the health sector there were around 325,000 apps available in app stores in 2017. Many of these applications are developed without the involvement of experts in the field and without a clear process for validating and evaluating them. So, just as m-Health can help patient safety, it can act in the opposite way, by offering information that is not based on scientific evidence8. It is therefore important to evaluate the available applications before using or recommending them.

Given the need to promote research and technological innovations that can contribute to patient safety, a scoping review is needed to map the applications developed for this thematic area. It is believed that this survey will be able to identify gaps and indicate ways to build technologies that can advance on existing models.

On December 21, 2021, a search was carried out in the Cinahl, Pubmed and Open Science databases in order to identify scoping reviews with the same objective and no records were found.

Therefore, this review aims to map scientific production related to patient safety applications.

METHOD

This is a scoping review, based on the method proposed by the JBI9 and the recommendations of the Preferred Reporting Items for Systematic Review and Meta-Analyses for Scoping Reviews (Prisma SCR) - Reference with research protocol registered in the Open Science Framework (https://osf.io/8b9pz/). It was carried out in five stages: identification of the research question; identification of relevant studies; selection of studies; data analysis; grouping, synthesis and presentation of the data10.

The research question was outlined using the strategy P (Participants= patients, health professionals or health students), C (Concept= mobile applications) and C (Context= patient safety). Thus, the question to be answered by this review was: what patient safety apps and their features are available to patients, students and healthcare professionals?

In order to identify the relevant studies, keywords and Meshs that covered the subject of this review, an initial search was carried out in the indexed databases PUBMED and Cumulative Index to Nursing and Allied Health Literature (CINAHL). After this stage, with the help of a librarian, the search strategies for each database were defined using a combination of the Boolean operators AND and OR (Chart 1).

The data for the review was collected between February and March 2022 from the Science Direct, PUBMED, Embase, Latin American and Caribbean Health Sciences Literature (LILACS) and Cumulative Index to Nursing and Allied Health Literature (CINAHL) databases. The bases used to evaluate the gray literature were: Brazilian Digital Library of Theses and Dissertations (BDTD), Capes Catalog of Theses and Dissertations, Networked Digital Library of Theses and Dissertations (NDLTD), E-Theses online service (Ethos), Latin American Repository Network and the Portuguese Open Access Scientific Repository (ACAAP). The search for property records took place in the offices: National Institute of Industrial Property (INPI), World Intellectual Property Organization (WIPO), United States Patent and Trademark Office (USPTO) and Espacenet and Latipati.

The combinations "Patient Safety" AND "Mobile Applications", "Patient Safety" AND "Mobile Applicatons" were used in the evaluation of gray literature and intellectual property registration.

Chart 1 -
Search strategy by database consulted. São Paulo, SP, Brazil, 2022.

Database	Search strategy
SCOPUS	(TITLE-ABS-KEY ( "patient safety") AND TITLE-ABS-KEY ("Mobile Applications" OR "Mobile Application" OR "Mobile Apps" OR "Mobile App" OR "Portable Electronic App" OR "Portable Electronic Applications") )
EMBASE	'patientsafety':ti,ab,kw AND 'mobile application':ti,ab,kW
LILACS	Patient Safety and Mobile Applications
PUBMED	("patient safety"[MeSH Terms] AND ("Mobile Applications"[MeSH Terms] OR "Mobile Application"[All Fields] OR "Mobile Apps"[All Fields] OR "Mobile App"[All Fields] OR "Portable Electronic Apps"[All Fields] OR "Portable Electronic Applications"[AllFields])
CINAHL	MH patient safety AND AB ( mobile applications or apps or mobile apps or mhealth or ehealth )
SCIENCE DIRECT	"patient safety" AND ("mobile applications")

Original studies, gray literature, patents and registrations of computer programs produced in any language, aimed at patients, students and health professionals, published between 2005 and 2021, that address the description of the construction and/or use of applications aimed at patient safety, linked to one of the international patient safety goals, were included: (1) identify the patient correctly; (2) improve the effectiveness of communication; (3) improve the safety of high-surveillance medications; (4) ensure surgeries with the correct intervention site, correct procedure and correct patient; (5) reduce the risk of healthcare-associated infections; (6) reduce the risk of harm to the patient from falls. For the goal related to medication, any type of medication was considered. The time frame was proposed as a result of the publication in 2005 of the first global patient safety challenge “Clean Care is Safer Care”, involving actions related to improving Hand Hygiene in Health Services11.

Reviews, editorials, annals, theoretical essays, single case studies, games and apps with no direct relation to one of the patient safety goals, such as apps aimed at clinical disease management, were excluded.

After selection, the sample comprised 27 studies found in the databases and one application identified in the intellectual property registration offices (Figure 1). Data was extracted from the articles and gray literature using a tool developed by the authors containing important information for evaluating the articles and applications.

The following information was extracted from reading the studies: article’s title, year of publication, country, objective of the study and the app, compatibility, functionalities of the app, information available, target audience, information on the method of building the app, tests used and test audience, results, international patient safety target. To extract data from patents and computer registrations, the following information was extracted: title, country, year of registration, available information, target audience, international patient safety target.

The selection and extraction process was carried out by two reviewers and, in the event of disagreements, a third reviewer was responsible for a new evaluation. The articles were subjected to a reading of titles and abstracts and the pre-selected articles were read in full. A concordance test was carried out between the reviewers to ensure the effectiveness of the data collection instrument and to align the selection with the evaluation of 15 manuscripts.

The data was analyzed and categorized descriptively and grouped according to the safety goal addressed, presenting the relevant information in the form of tables. The review was constructed according to the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR).

As this was a review using public domain data, it was not necessary to submit the study to the Research Ethics Committee.

Figure 1.
PRISMA flow diagram.

RESULTS

813 references were found in the databases and three duplicate evaluations were carried out, the first using the conventional Endnote reference management system, the second by evaluating the name of the article and the third by the authorship of the article, leaving 609 references. After analyzing the titles and abstracts, 35 articles were read in full. In the end, 23 articles were selected, 12 of which were excluded for various reasons, such as: not being an original article or application, not being in line with one of the international patient safety goals. The search strategy in the gray literature resulted in 32 studies and, after excluding duplicates, five studies went on to full text evaluation. Of these, four studies made up the sample (Figure 1).

In the search for prior art in intellectual property databases, 133 records were obtained, of which seven were evaluated in full and five were excluded because they were not an application or were not in line with the goals defined in the scope of this review. Only the "SafeCare - Management for Patient Safety"12 application was included, which presented various patient safety protocols for healthcare professionals to consult (hand hygiene, fall prevention, patient identification, use, administration and prescription of medication, communication between healthcare professionals and some specific clinical protocols). The app was not available in the Play Store and Apple Store at the time of the research, so it was not possible to fully evaluate it.

Chart 2 -
Characteristics of the included studies, safety goal, target audience and operating system for making the application available. São Paulo, SP, Brazil, 2022.

Author	Country	Type of study	Security theme addressed	Target population	Operating system
Yap et al, 201213	Singapore	Methodological	Medication	Health professionals	IOS
Mira et al, 201414	Spain	Randomized Clinical Trial	Medication	Older patients	IOS
Mira et al, 201515	Spain	Methodological	Medication	Older patients with polypharmacy	IOS and Android
Buning et al, 201616	Netherlands	Methodological	Medication	Patients	IOS
Wentzel et al, 201617	Netherlands	Methodological	Medication	Nurses	Not reported
Marien et al, 201818	Belgium	Methodological	Medication	Patients	Not reported
Baumann et al, 201919	Germany	Methodological	Medication	Health Professionals	Not reported
Ankem et al, 201920	USA	Methodological	Medication	Patients, caregivers and physicians	IOS
Nedovic et al, 201921	Switzerland	Methodological	Medication	Visually impaired patients	IOS and Android
Taber et al, 201922	USA	Methodological	Medication	Health professionals and kidney transplant patients	Not reported
Santos, 201923	Brazil	Methodological	Medication	Nurses	Android
Pereira, 201924	Brazil	Methodological	Medication	Patient	Android
Madrigal-Cadavid et al, 202025	Colombia	Methodological	Medication	Visually impaired patients	Android
Holden et al, 202026	USA	Methodological	Medication	Older patients	Android
Misiak Caldas et al, 202027	Brazil	Methodological	Medication	Health professionals	Not reported
Aldughayfiq, Sampalli, et al, 202128	Canada	Methodological	Medication	Patients	Android
Siebert et al, 202129	Switzerland	Randomized Clinical Trial	Medication	Paramedics	Not reported
Beck et al, 201830	Germany	Randomized Clinical Trial	Safe Surgery	Health Professionals	Not reported
Molina et al, 201731	USA	Methodological	Safe Surgery	Physicians	IOS
Betancourt et al, 201732	Colombia	Clinical Trial with Intervention	Safe Surgery	Health professionals	IOS and Android
Russ et al 202033	United Kingdom	Methodological	Safe Surgery	Surgical patients	IOS
LindI et al, 202034	Sweden	Methodological	Identification	Health professionals	Android
Jeonet al, 201935	Korea	Methodological	Identification	Health professionals	Android
Flohr et al, 201836	Mexico	Methodological	Communication	Health professionals	Not reported
Schmidt et al, 201937	Switzerland	Methodological	Communication	Health professionals	Not reported
Barbosa, 202138	Brazil	Methodological	Communication	Health professionals	Not reported
Han et al., 202039	Korea	Experimental study (pre and post-test group)	Prevention of falls	Pediatric patients and caregivers	Not reported

Most of the original articles evaluated were produced in the last five years (16; 69.5%), with the countries in particular: United States of America (USA) (4; 17.3%), Brazil (4; 17.3%) and Switzerland (3; 13.4%). There was a predominance of methodological studies (22; 81.4%), followed by clinical trials (04; 14.81%) (Chart 2).

Of the apps identified, most (17; 62.9%) are related to the goal of safe medication, followed by safe surgery (4; 14.81%), safe communication (3; 11.1%), correct identification (2; 7.4%) and multiple goals (1; 3.7%) (Chart 2).

Apps aimed at the goal of safe medication began to be produced in 2012 and their main focus is on patients 14^-16^,18^,21^,24^-26^,28 (9; 52.9%), including special populations such as older adults14^,15 and the visually impaired15^-16; healthcare professionals13^,17^,19^,23^,27^,29 (7; 41.1%); or a combination of both20^-22 (professionals and patients) (2; 11.7%) (Chart 2). The main function of the apps aimed at patients was to record the medications used15^,18^,20^-21^,24, remind patients to take them and educate them about possible interactions14^,20^,21^,22, storage14^-15^,18 and side effects15^,22. For health professionals, these apps are aimed at consulting medication interactions13, calculating doses17^,19^,29 and educating people to avoid medication errors17^,23^,27 (Chart 3)..

The apps that address the topic of safe surgery (4; 14.8%) began to be produced in 2012, aimed at different audiences (patients33, healthcare professionals32 anesthesia residents30 and physicians31), and are geared towards patient education33, preventing the retention of surgical materials31 and training residents30 and healthcare staff32 (Chart 3).

In the area of effective communication, apps began to be released in 2018 and are aimed at optimizing communication between healthcare staff36^-38. Fall prevention goals38 were addressed by one study and patient identification was addressed by two34^-35, produced as of 2019.

The operating system was not reported in 1117^-19^,22^,27^,29^-30^,36^-39 (40.7%) of the literature evaluated and there was similarity in the number of apps available for the Android operating system23^-26^,28^,34^-35 (7; 25.9%) and IOs13^-14^,16^,20^,31^,33 (6; 22.1%). Most of the studies13^-15^,20^-28^,30^-33^,35^-39 (21; 77.7%) describe the process of building the application in terms of content selection or software architecture. As for its usability evaluation, forms constructed by the researchers13^,15^,18^,20^-21^,25^,28^,33^,35 or validated scales such as the System Usability Scale (SUS)16^,26^,38 were used, by means of simulation tests or before and after tests on the use of the APP.

In the review, it was possible to identify that the process of building apps involved a multi-professional team in some studies26^-27^,33^-34^,36 and that user-centered design was one of the methods used14^,17^,25^-27. In addition, in order to build the content of the applications, studies report that literature reviews were carried out 13^,21^,25^,39. The evaluations of efficiency, quality and effectiveness of the APPs do not show a uniform pattern, and for usability the most common tool was SUS 16^,26^,38. The tools used to build the APPs were described in three articles: adobe experience design27, ionic337, adobe dreamweaver CS413.

The simulation and clinical study tests carried out on the apps showed that their use was positive in helping to achieve the safety goals 17^,29^,33. In qualitative evaluations carried out with semi-structured interviews and meetings, users reported that they believed the introduction of technology could help mitigate medication errors14^,28, described that the use of the app could provide greater patient engagement26^,33^,39 and were willing to use the devices15^,21. Simulations and clinical studies have shown an improvement in decision-making13, learning30 and a reduction in ureteral stent losses31

Chart 3 -
Characteristics of the included studies, application name, application objective, functionalities. São Paulo, SP, Brazil, 2022.

Author	Application name	Application objective	Features
Medication
Aldughayfiq, Sampalli, et al 202128	Not reported	Optimize the prescription process and the process of dispensing prescription medications.	Electronic prescription tracking; biometric authentication; Near Field Communication - (NFC) prescription transfer.
Balman et al, 201919	Not reported	Providing support in calculating and administering medication.	Consultation and information on medications.
Madrigal-Cadavid et al., 202025	Not reported	Facilitate access to medication information.	Consultation and presentation of information Voice assistant, vibration, sound alerts, font size adjustments, non-text alterative text, verbalization of warnings and talkback. Location of pharmacies and hospitals.
Holden et al, 202026	Brain Buddy	Inform and train older adults to consider the risks and benefits of anticholinergics.	Texts, videos and test to calculate personal risk score for anticholinergic use.
Mira et al, 201414	Alice	To remind people to use medications in the correct doses, to distinguish between medications, to avoid possible known interactions and errors in the use of medications. Teaching about the storage of medication.	Stores prescriptions and related instructions, images of medications and prescribers' recommendations. Interaction by text message - photo - use of text - alarms.
Buning et al., 201616	Mymedication	Creation of a list of medications used.	Scanning of the barcode on medication packaging; and information on the name and dosage of the medication compared with a database of medications included in the application. Manual recording of information. Medication reminder alarms.
Ankem et al, 201920	Scopia RX	Manage patient medication and provide up-to-date, personalized information such as proper dosage and possible medication interactions.	It analyzes incoming medications and issues alerts for discrepancies, such as incompatibilities or incorrect dosages.
Nedovic et al, 201921	MyPills	Medication management for blind people.	Identification of the medication packaging. Voice output of the name of the medication and intake schedule. Storage of medication and dosage information. Code display allows the current medication to be transferred to another healthcare professional. Prescription renewal reminder.
Mira et al, 201515	Tumedicine	Transforming Ean-13 and QR codes associated with medication into verbal instructions, to enable safer use of medication by patients.	Store information about the medication (purpose, dose, side effects, precautions, expiry date, storage). Reading the medication box code. Offering audio information.
Taber et al, 201922	Not reported	Monitoring transplant patients for medication safety.	Notices for checking blood pressure, blood glucose, recording medication intake times, research into adverse medication events and their severity.
Siebert et al, 202129	PedAmines	Providing information on the preparation and administration of medications.	Information on the preparation and administration of medications used in cardiopulmonary resuscitation, according to the patient's weight and age.
Yap et al, 201212	OncoRX-Mi	To detect alternative drug-treatment interactions (DCIS) between chemotherapy regimens (CRegs) and complementary and alternative medicines (CAMs).	Search for interactions based on the acronyms of the CRegs and the common name of the CAMs. Database with 2750 pairs of interactions between CRegs and CAMs, totaling 4408 DCI pairs.
Wentzel et al, 201617	Antibiotic APP	Antibiotic information to support nurses' work.	Instructions for administering or preparing parenteral antibiotics, information on side effects, allergies or the medication's action mechanisms.
Marien et al, 201818	Not reported	Documenting lists of medications.	Record of the medications used, how they were used, whether they were appropriate or not, and the prescriber.
Misiak Caldas et al, 202027	PrevMed	Train the healthcare team in the use of a protocol for preventing potentially dangerous medication errors.	Information through texts on Medical prescriptions; Storage, separation and distribution of medications; Preparation of medications; Administration of medications; Post-administration care.
Santos, 201923	Not reported	Orientation on the administration of antineoplastic chemotherapy for nurses.	Information on the routes of administration of the main medications, precautions at the time of administration, possible adverse effects, post-administration precautions and the nurse's conduct in the event of extravasation of a particular chemotherapy.
Pereira, 201924	MEDPAD	Enable patients to participate in the medication safety process in the hospital environment.	Access registration for professionals and patients; capture and digitalization of the prescription; movement in the prescription, scheduling and dispensing; confirmation - the patient will be able to follow the list of medications and confirm whether or not it has been administered, with the respective justification.
Prevenção de quedas
Han et al, 202039	APP SK	Preventing safety incidents among hospitalized children aged 3 to 6.	Games, audios, texts and tests.
Cirurgia Segura
Russ et al, 202033	Mysurgery	Empower patients to optimize the safety of care during surgery.	Interaction by message, informative texts, videos.
Molina et al, 201731	StentTracker	Improve patient safety, facilitate data collection and provide an efficient interface to simplify urethral stent tracking.	Provides procedure information, scheduled removal dates and product description for Boston Scientific ureteral stents.
Beck et al, 201830	Not reported	Present an audiovisual checklist to support anesthesia residents in the process of induction of anesthesia.	The checklist asks whether the following items are correct: Identification, procedure and location, presence of allergies, risk of difficult route, availability of materials and medication.
Kaltsidou et al, 201732	Not reported	Identify a critical event early during surgery and provide solutions for coping.	Flowchart that allows early identification of a critical event during surgery and, once detected, provides the steps for dealing with it.
Identificação do paciente
Jeon et al, 201935	Not reported	Facial identification of patients before procedures.	Facial biometrics and access to medical records.
LindI et al, 202034	Not reported	Make patient identification easier and quicker for hospital or nursing home staff.	Scanning the tag on the patient's ID bracelet with the mobile device's NFC reader.
Comunicação eficaz
Flohr et al, 201836	VitalPed	Solving communication and workflow problems.	Integrated display of vital parameters for single and multiple patients, intelligent alarms and reminders, team communication tool.
Schmidt et al, 201937	Not reported	To help health professionals structure their information for notification and on-call duty.	Create, store, modify and delete an ISBAR note.
Barbosa, 202138	Not reported	Assisting with the shift change in an emergency room.	SBAR-based shift handover tool and NEWS patient risk assessment scale.

DISCUSSION

The aim of this study was to map mobile applications aimed at patient safety according to the international goals of the World Health Organization. The goal most covered by the apps found in this review was that related to preventing medication errors, followed by safe surgery and effective communication. What is striking is the small number of findings related to the other goals and the absence of applications aimed at the goal “Reduce the risk of healthcare-associated infections”, which was the theme of the WHO's first global challenge11.

Applications aimed at preventing medication errors are in line with the WHO's 3rd global challenge, Medication without Harm, launched in 201740, which aims to reduce adverse events by at least 50% by 2022, using strategies that make the stages of the medication process more efficient. The third challenge involves developing technologies and empowering patients and their families to prevent medication errors40. The software analyzed seeks to involve patients in the process of safe medication, making it possible to set up a list of the medications used and offering information on use, dosage, adverse effects, storage, as well as helping patients to remember to take their medication. It is worth highlighting the possibility of these tools contributing to the transition of care, which is also envisaged by the challenge14^,16^,18^-21^,24^-26^,28.

For healthcare professionals, the findings regarding medication error prevention apps are primarily related to continuing education13^,17^,19^-20^,23^,27^,29, which contributes to error prevention by giving professionals quick and easy access to information about medications. One of the great advantages of developing mobile technologies is the possibility of optimizing health education actions, since applications allow information to be consulted at the desired frequency, anywhere, via a smartphone41. Thus, the strategy of using mobile applications for continuing education can be extended to other goals and also in the process of training health students, a population not covered in the studies analyzed.

The development of specific apps for older adults14^-15^,26 is something that deserves attention, given the ageing of the world's population. According to IBGE data, 67% of the older adults use a mobile phone and this number is on the rise42. The creation of software for the older adults enables the digital inclusion of this public and allows them autonomy and independence in caring for their own health43. The apps identified in this review explored the area of medication safety, and the descriptors used did not find any apps specifically aimed at preventing falls, which is an important point to work on with this population.

Communication-related applications36^-38 are designed to optimize the work process of healthcare teams, improving the flow of information between team members and thus preventing adverse events. The use of standardized communication processes has been encouraged by international organizations because it contributes to patient safety and improves the quality and effectiveness of care44.

Two of the applications used the SBAR (Situation-Background-Assessment-Recommendation) instrument recommended by the WHO and the Institute for Healthcare Improvement45. Applications that improve professional-patient communication are emerging as a possible innovation to be developed.

For the goal of safe surgery, the applications found30^-33 had the following objectives: to prevent ureteral stents from being forgotten, to train the team to identify and intervene in critical events that could impact on the safety of care, and to promote patient education. Apps related to patient safety can be an alternative to improve patients' self-management and provide access to information that improves their knowledge, which can act as barriers to preventing adverse events46.

The search for information by the population has been growing and, with the onset of the Covid-19 pandemic, there has been a 50% increase in downloads of health apps47. This information can be a worrying aspect when we identify the lack of regulatory control over the process of construction, content and evaluation of software. The information offered to patients is not monitored, which can have a negative impact on patient health and safety48.

This review found that the devices built provided information on software architecture, content selection and the tests used. The most frequently described construction method was User-Centered Design, which focuses on the participation of the client in the construction of the software, placing them at the center of the development of the design, content and usability project49. There is heterogeneity in the method of selecting the evidence, the construction platform and the evaluation by those responsible for developing the software. Among the most common tests used to evaluate applications is usability, which refers to efficiency of use and ease of use50.

However, in this regard there was also variability in the instruments used. The lack of standardization in the creation, security and validity of the application makes the process of replicating studies challenging. A similar result was found in another study, which reaffirms that the use of varied instruments in the creation and evaluation of software makes it difficult to make a valid comparison and reproduce research51. The International Organization for Standardization (ISO) and International Electrotechnical Commission instrument called ISO/IEC 2501052 describes the necessary requirements for software: functional suitability; reliability; usability; performance efficiency; compatibility; security; maintainability, portability. However, it is not a legislative document and no reference was found to its use in the apps evaluated.

The scenario of using mobile technologies has grown exponentially, strengthened by the social isolation imposed by the Covid-19 pandemic, and tends to consolidate itself, as well as the consumption of health apps, which is the new modus operandis of doing health53.

The research carried out indicates that there is a vast field to be explored with the development of new applications that can contemplate the patient's international goals. However, there is a need for legislation to regulate the veracity of the information offered and the safety of app traffic, conditions that were little explored in the studies analyzed, and to strengthen the involvement of the APP's target population and health professionals in its construction and validation.

CONCLUSION

This review identified applications related to the main international patient safety targets and surveyed their main characteristics. The prevention of medication errors was addressed by most of the apps and there were gaps in relation to goals such as hand hygiene and falls prevention. The medication apps were focused primarily on patients in order to help them manage their medication use. For the other targets, the focus was on health professionals. It was found that there is no regulatory framework for the certification of healthcare applications, nor is there a standard for their development and validation, which makes it difficult to replicate studies.

REFERENCES

• 1. Institute of Medicine (US) Committee on Quality of Health Care in America; Kohn LT, Corrigan JM, Donaldson MS, editors. To err is human: Building a safer health system. Washington (DC): National Academies Press; 2000.
• 2. Silva LLT, Dias FCS, Maforte NTP, Menezes AC. Patient safety in Primary Health Care: Perception of the nursing team. Esc Anna Nery [Internet]. 2022 [cited 2023 Jul 18];26:e20210130. Available from: https://doi.org/10.1590/2177-9465-EAN-2021-0130
  » https://doi.org/10.1590/2177-9465-EAN-2021-0130
• 3. Brazil. Ministério da Saúde. Portaria n. 529 de 1º de abril de 2013 (BR). Institui o Programa Nacional de Segurança do Paciente (PNSP) [Internet]. Diário Oficial da União, Brasília, 2013 [cited 2023 Jul 18]. Available from: https://bvsms.saude.gov.br/bvs/saudelegis/gm/2013/ prt0529_01_04_2013.html
  » https://bvsms.saude.gov.br/bvs/saudelegis/gm/2013/ prt0529_01_04_2013.html
• 4. World Health Organization (WHO). Global patient safety action plan 2021-2030: towards eliminating avoidable harm in health care [Internet]. Geneva, (CH): World Health Organization; 2021 [cited 2023 Jul 18]. 86 p. Available from: https://www.who.int/publications/i/item/9789240032705
  » https://www.who.int/publications/i/item/9789240032705
• 5. Lima Neto AV, Silva MF, Santos VEP. Contribuições das tecnologias em saúde para a segurança do paciente. Rev Cubana Enferm [Internet]. 2019 [cited 2023 Jul 18];35(4):e2125. Available from: http://www.revenfermeria.sld.cu/index.php/enf/article/view/2125
  » http://www.revenfermeria.sld.cu/index.php/enf/article/view/2125
• 6. Gomes ATL, Assis YMS, Ferreira LL, Bezerril MS, Chiavone FBT, Santos VEP. Technologies applied to patient safety: A bibliometric review. Rev Enferm Cent-Oeste Min [Internet]. 2017 [cited 2023 Jul 18];7:1-11. Available from: https://pesquisa.bvsalud.org/portal/resource/pt/biblio-908340
  » https://pesquisa.bvsalud.org/portal/resource/pt/biblio-908340
• 7. Moss RJ, Süle A, Kohl S. eHealth and mHealth. Eur J Hosp Pharm [Internet]. 2019 [cited 2023 Jul 18];26(1):57-8. Available from: https://ejhp.bmj.com/content/26/1/57
  » https://ejhp.bmj.com/content/26/1/57
• 8. Akbar S, Coiera E, Magrabi F. Safety concerns with consumer-facing mobile health applications and their consequences: A scoping review. J Am Med Inform Assoc [Internet]. 2020 [cited 2023 Jul 18];27(2):330-40. Available from: https://doi.org/10.1093/jamia/ocz175
  » https://doi.org/10.1093/jamia/ocz175
• 9. Peters MDJ, Godfrey C, McInerney P, Munn Z, Tricco AC, Khalil H. Chapter 11: Scoping Reviews (2020 version). In: Aromataris E, Munn Z. JBI Manual for Evidence Synthesis, JBI [Internet]. 2020 [cited 2023 Jul 18]. Available from: https://synthesismanual.jbi.global
  » https://synthesismanual.jbi.global
• 10. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ [Internet]. 2021 [cited 2023 Jul 18];372(71):n71. Available from: https://doi.org/10.1136/bmj.n71
  » https://doi.org/10.1136/bmj.n71
• 11. World Health Organization (WHO). Global patient safety challenge: Clean care is safer care [Internet]. Genebra, (CH): World Health Organization; 2005 [cited 2023 Jul 18]. Available from: https://www.who.int/news-room/events/detail/2005/10/13/default-calendar/clean-care-is-safer-care
  » https://www.who.int/news-room/events/detail/2005/10/13/default-calendar/clean-care-is-safer-care
• 12. Yasojima EY, Pinheiro LD, Chaves RO. Safercare: gestão para a segurança do paciente. REAS [Internet]. 2019 [cited 2023 Jul 18];31 Suppl 31:e1075. Available from: https://doi.org/10.25248/ reas.e1075.2019
  » https://doi.org/10.25248/ reas.e1075.2019
• 13. Yap KYL, See CS, Kuo EY, Chui WK, Chan A. Utilizing mobile networks for the detection of clinically relevant interactions between chemotherapy regimens and complementary and alternative medicines. J Altern Complement Med [Internet]. 2012 [cited 2023 Jul 18];18(2):165-74. Available from: https://doi.org/10.1089/acm.2010.0846
  » https://doi.org/10.1089/acm.2010.0846
• 14. Mira JJ, Navarro I, Botella F, Borrás F, Nuño-Solinís R, Orozco D, et al. A spanish pillbox app for elderly patients taking multiple medications: Randomized controlled trial. J Med Internet Res [Internet]. 2014 [cited 2023 Jul 19];16(4):e99. Available from: https://doi.org/10.2196/jmir.3269
  » https://doi.org/10.2196/jmir.3269
• 15. Mira JJ, Guilabert M, Carrillo I, Fernández C, Vicente MA, Beltrán DO, et al. Use of QR and EAN-13 codes by older patients taking multiple medications for a safer use of medication. Int J Med Inform [Internet]. 2015 [cited 2023 Jul 19];84(6):406-12. Available from: https://doi.org/10.1016/j. ijmedinf.2015.02.001
  » https://doi.org/10.1016/j. ijmedinf.2015.02.001
• 16. Buning AW, Klopotowska JE, Duyvendak M, Engelen LJLPG, Arts J. Patient empowerment through provision of a mobile application for medication reconciliation: A proof of concept study. BMJ Innov [Internet]. 2016 [cited 2023 Jul 19];2(4):152-7. Available from: https://doi.org/10.1136/ bmjinnov-2015-000110
  » https://doi.org/10.1136/ bmjinnov-2015-000110
• 17. Wentzel J, Van Drie-Pierik R, Nijdam L, Geesing J, Sanderman R, Van Gemert-Pijnen JEWC. Antibiotic information application offers nurses quick support. Am J Infect Control [Internet]. 2016 [cited 2023 Jul 19];44(6):677-84. Available from: https://doi.org/10.1016/j.ajic.2015.12.038
  » https://doi.org/10.1016/j.ajic.2015.12.038
• 18. Marien S, Legrand D, Ramdoyal R, Nsenga J, Ospina G, Ramon V, et al. A web application to involve patients in the medication reconciliation process: A user-centered usability and usefulness study. J Am Med Inform Assoc [Internet]. 2018 [cited 2023 Jul 19];25(11):1488-500. Available from: https://doi.org/10.1093/jamia/ocy107
  » https://doi.org/10.1093/jamia/ocy107
• 19. Baumann D, Dibbern N, Sehner S, Zöllner C, Reip W, Kubitz JC. Validation of a mobile app for reducing errors of administration of medications in an emergency. J Clin Monit Comput [Internet]. 2019 [cited 2023 Jul 19];33(3):531-9. Available from: https://doi.org/10.1007/s10877-018-0187-3
  » https://doi.org/10.1007/s10877-018-0187-3
• 20. Ankem K, Cho S, Simpson D. Nurses’ perceptions and problems in the usability of a medication safety app. Inform Health Soc Care [Internet]. 2019 [cited 2023 Jul 19];44(1):48-69. Available from: https://doi.org/10.1080/17538157.2017.1364249
  » https://doi.org/10.1080/17538157.2017.1364249
• 21. Nedovic T, Umeri-Sali N, Denecke K. Supporting blind and visually impaired persons in managing their medication. Stud Health Technol Inform [Internet]. 2019 [cited 2023 Jul 19];267:189-96. Available from: https://doi.org/10.3233/SHTI190826
  » https://doi.org/10.3233/SHTI190826
• 22. Taber DJ, Pilch NA, McGillicuddy JW, Mardis C, Treiber F, Fleming JN. Using informatics and mobile health to improve medication safety monitoring in kidney transplant recipients. Am J Health Syst Pharm [Internet]. 2019 [cited 2023 Jul 19];76(15):1143-9. Available from: https://doi.org/10.1093/ajhp/zxz115
  » https://doi.org/10.1093/ajhp/zxz115
• 23. Santos, TO. Criação de um aplicativo de orientação aos enfermeiros para administração de antineoplásicos [dissertação] [Internet]. Florianópolis, SC(BR): Universidade Federal de Santa Catarina; 2019 [cited 2023 Jul 19]. 68 p. Available from: https://repositorio.ufsc.br/handle/123456789/215242?show=full
  » https://repositorio.ufsc.br/handle/123456789/215242?show=full
• 24. Pereira FGF. Medpad: tecnologia para segurança medicamentosa com a participação do paciente [tese] [Internet]. Fortaleza, CE(BR): Universidade Federal do Ceará; 2019 [cited 2023 Jul 19]. 160 p. Available from: http://www.uece.br/ppcclis/wp-content/uploads/sites/55/2020/05/ TESE-GILBERTO.pdf
  » http://www.uece.br/ppcclis/wp-content/uploads/sites/55/2020/05/ TESE-GILBERTO.pdf
• 25. Madrigal-Cadavid J, Amariles P, Pino-Marín D, Granados J, Giraldo N. Design and development of a mobile app of drug information for people with visual impairment. Res Social Adm Pharm [Internet]. 2020 [cited 2023 Jul 19];16(1):62-7. Available from: https://doi.org/10.1016/j.sapharm.2019.02.013
  » https://doi.org/10.1016/j.sapharm.2019.02.013
• 26. Holden RJ, Campbell NL, Abebe E, Clark DO, Ferguson D, Bodke K, et al. Usability and feasibility of consumer-facing technology to reduce unsafe medication use by older adults. Res Social Adm Pharm [Internet]. 2020 [cited 2023 Jul 19];16(1):54-61. Available from: https://doi.org/10.1016/j.sapharm.2019.02.011
  » https://doi.org/10.1016/j.sapharm.2019.02.011
• 27. Caldas MM, Tourinho FSV, Radünz V, Fermo VC, Ilha P, Alves TF. Aplicativo móvel para prevenção de erros de medicação: PREVMED. Cienc Enferm [Internet]. 2020 [cited 2023 Jul 20];26(4):1-9. Available from: https://doi.org/10.4067/s0717-95532020000100401
  » https://doi.org/10.4067/s0717-95532020000100401
• 28. Aldughayfiq B, Sampalli S. A framework to lower the risk of medication prescribing and dispensing errors: A usability study of an NFC-based mobile application. Int J Med Inform [Internet]. 2021 [cited 2023 Jul 20];153:104509. Available from: https://doi.org/10.1016/j.ijmedinf.2021.104509
  » https://doi.org/10.1016/j.ijmedinf.2021.104509
• 29. Siebert JN, Bloudeau L, Combescure C, Haddad K, Hugon F, Suppan L, et al. Effect of a mobile app on prehospital medication errors during simulated pediatric resuscitation: A randomized clinical trial. JAMA Netw Open [Internet]. 2021 [cited 2023 Jul 20];4(8):e2123007. Available from: https://doi.org/10.1001/jamanetworkopen.2021.23007
  » https://doi.org/10.1001/jamanetworkopen.2021.23007
• 30. Beck S, Reich C, Krause D, Ruhnke B, Daubmann A, Weimann J, et al. For beginners in anaesthesia, self-training with an audiovisual checklist improves safety during anaesthesia induction: A randomised, controlled two-centre study. Eur J Anaesthesiol [Internet]. 2018 [cited 2023 Jul 20];35(7):527-33. Available from: https://doi.org/10.1097/EJA.0000000000000781
  » https://doi.org/10.1097/EJA.0000000000000781
• 31. Molina WR, Pessoa R, Donalisio SR, Kenny MCC, Gustafson D, Nogueira L, et al. A new patient safety smartphone application for prevention of “forgotten” ureteral stents: Results from a clinical pilot study in 194 patients. Patient Saf Surg [Internet]. 2017 [cited 2023 Jul 20];11(1):10. Available from: https://doi.org/10.1186/s13037-017-0123-3
  » https://doi.org/10.1186/s13037-017-0123-3
• 32. Kaltsidou SAB, Volpe AML, Ramírez CPN, Ruiz GO, Gaviria MCV, Gómez RF, et al. Applications (APPS), an innovative health trend: development of a mobile application for the management of critical events during orthopaedic and trauma surgery. Rev Col Or y Tra [Internet]. 2017 [cited 2023 Jul 20];31(4):211-6. Available from: https://doi.org/10.1016/j.rccot.2017.06.001
  » https://doi.org/10.1016/j.rccot.2017.06.001
• 33. Russ S, Latif Z, Hazell AL, Ogunmuyiwa H, Tapper J, Wachuku-King S, et al. A smartphone app designed to empower patients to contribute toward safer surgical care: Community-based evaluation using a participatory approach. JMIR Mhealth Uhealth [Internet]. 2020 [cited 2023 Jul 20];8(1):e12859. Available from: https://doi.org/10.2196/12859
  » https://doi.org/10.2196/12859
• 34. Lind I, Jeri PC. An analysis in patient safety: Alternative patient identification using a mobile applicatio [tese] [Internet]. Stocolomo, (SE): Skolan för kemi, bioteknologi och hälsa; 2020 [cited 2023 Jul 20]. 48 p. Available from: http://www.diva-portal.org/smash/get/diva2:1448165/FULLTEXT01.pdf
  » http://www.diva-portal.org/smash/get/diva2:1448165/FULLTEXT01.pdf
• 35. Jeon B, Jeong B, Jee S, Huang Y, Kim Y, Park GH, et al. A facial recognition mobile app for patient safety and biometric identification: design, development, and validation. JMIR Mhealth Uhealth [Internet]. 2019 [cited 2023 Jul 20];7(4):e11472. Available from: https://doi.org/10.2196/11472
  » https://doi.org/10.2196/11472
• 36. Flohr L, Beaudry S, Johnson KT, West N, Burns CM, Ansermino JM, et al. Clinician-driven design of vital pad-an intelligent monitoring and communication device to improve patient safety in the intensive care unit. IEEE J Transl Eng Health Med [Internet]. 2018 [cited 2023 Jul 20];6:3000114. Available from: https://doi.org/10.1109/JTEHM.2018.2812162
  » https://doi.org/10.1109/JTEHM.2018.2812162
• 37. Schmidt T, Kocher DR, Mahendran P, Denecke K. Dynamic pocket card for implementing ISBAR in shift handover communication. Stud Health Technol Inform [Internet]. 2019 [cited 2023 Jul 20];3(267):224-9. Available from: https://doi.org/10.3233/SHTI190831
  » https://doi.org/10.3233/SHTI190831
• 38. Barbosa IS, Jaques AE, Radovanovic CAT, Andrade L de, Dermatte LPG, Souza CM de, et al. Development of a mobile application for emergency shift handovers using the National Early Warning Score. Rev Gaúcha Enferm [Internet]. 2023 [cited 2023 Jul 20];44:e20220130. Available from: https://doi.org/10.1590/1983-1447.2023.20220130.en
  » https://doi.org/10.1590/1983-1447.2023.20220130.en
• 39. Han J, Oh WO, Park IT, Lee A. Development and effects of a mobile application for safety incident prevention among hospitalized Korean children: A pilot study of feasibility and acceptability. J Pediatr Nurs [Internet]. 2020 [cited 2023 Jul 20];51:e69-76. Available from: https://doi.org/10.1016/j.pedn.2019.09.022
  » https://doi.org/10.1016/j.pedn.2019.09.022
• 40. World Health Organization (WHO). Medication without harm: Global patient safety challenge on medication safety [Internet]. Geneva, (CH): World Health Organization; 2017 [cited 2023 Jul 20]. 16 p. Available from: https://iris.who.int/bitstream/handle/10665/255263/WHO-HIS-SDS-2017.6
  » https://iris.who.int/bitstream/handle/10665/255263/WHO-HIS-SDS-2017.6
• 41. Timmers T, Janssen L, Kool RB, Kremer JA. Educating patients by providing timely information using smartphone and tablet apps: Systematic review. J Med Internet Res [Internet]. 2020 [cited 2023 Jul 20];22(4):e17342. Available from: https://doi.org/10.2196/17342
  » https://doi.org/10.2196/17342
• 42. Instituto Brasileiro de Geografia e Estatística (IBGE). PNAD Contínua - Pesquisa Nacional por Amostra de Domicílios Contínua [Internet]. Diretoria de Pesquisas, Coordenação de Trabalho e Rendimento, Pesquisa Nacional por Amostra de Domicílios Contínua; 2018/2019 [cited 2023 Jul 19]. 12 p. Available from: https://biblioteca.ibge.gov.br/visualizacao/livros/liv101794_informativo.pdf
  » https://biblioteca.ibge.gov.br/visualizacao/livros/liv101794_informativo.pdf
• 43. Amorim DNP, Sampaio LVP, Carvalho GA, Vilaça KHC. Aplicativos móveis para a saúde e o cuidado de idosos. Reciis - Rev Eletron Comun Inf Inov Saúde [Internet]. 2018 [cited 2023 Jul 20];12(1):58-71. Available from: https://doi.org/10.29397/reciis.v12i1.1365
  » https://doi.org/10.29397/reciis.v12i1.1365
• 44. Hemesath MP, Kovalski AV, Echer IC, Lucena AF, Rosa NG. Effective communication on temporary transfers of inpatient care. Rev Gaúcha Enferm [Internet]. 2019 [cited 2023 Jul 20];40(spe):e20180325. Available from: https://doi.org/10.1590/1983-1447.2019.20180325
  » https://doi.org/10.1590/1983-1447.2019.20180325
• 45. Institute for Healthcare Improvement (IHI). Sbar Tool: Situation-background-assessment-recommendation [Internet]. Cambridge, MA(US): Institute for Health Care Improvement; 2018 [cited 2023 Jul 20]. Available from: http://www.ihi.org/resources/Pages/Tools/SBARToolkit.aspx
  » http://www.ihi.org/resources/Pages/Tools/SBARToolkit.aspx
• 46. Lee NJ, Ahn S, Lee M. The effects of a mobile application for patient participation to improve patient safety. Health Expect [Internet]. 2022 [cited 2023 Jul 20];25(4):1601-18. Available from: https://doi.org/10.1111/hex.13503
  » https://doi.org/10.1111/hex.13503
• 47. App Annie. The state of mobile 2021 [Internet]. data.ai; 2020 [cited 2023 Jul 20]. Available from: https://www.data.ai/de/go/state-of-mobile-2021
  » https://www.data.ai/de/go/state-of-mobile-2021
• 48. Kao C, Liebovitz DM. Consumer mobile health apps: Current state, barriers, and future directions. PM R [Internet]. 2017 [cited 2023 Jul 20];9(5S):S106-15. Available from: https://doi.org/10.1016/j. pmrj.2017.02.018
  » https://doi.org/10.1016/j. pmrj.2017.02.018
• 49. Azevedo PM de, Gibertoni D. A importância do design centrado no usuário em metodologias ageis como requisito de usabilidade. INFA [Internet]. 2020 [cited 2023 Oct 21];17(2):293-305. Available from: https://doi.org/10.31510/infa.v17i2.986
  » https://doi.org/10.31510/infa.v17i2.986
• 50. Dawson RM, Felder TM, Donevant SB, McDonnell KK, CardIII EB, King CC. What makes a good health “app”? Identifying the strengths and limitations of existing mobile application evaluation tools. Nurs Inq [Internet]. 2020 [cited 2023 Jul 20];27(2):e12333. Available from: https://doi.org/10.1111/nin.12333
  » https://doi.org/10.1111/nin.12333
• 51. Luz PK da, Machado R da S, Oliveira RKC de, Meneses M de O, Marques M do CMP, Andrade EMLR. Educational technologies devised to teach basic life support to adolescents: A scoping review. Texto Contexto Enferm [Internet]. 2023 [cited 2023 Oct 21];32:e20220332. Available from: https://doi.org/10.1590/1980-265X-TCE-2022-0332en
  » https://doi.org/10.1590/1980-265X-TCE-2022-0332en
• 52. International Organization for Standardization (ISSO). Systems and software engineering - Systems and software Quality Requirements and Evaluation (SQuaRE) - System and software quality models [Internet]. ISO/IEC 25010; 2011 [cited 2023 Jul 20]. 34 p. Available from: https://www.iso.org/standard/35733.html
  » https://www.iso.org/standard/35733.html
• 53. Celuppi IC, Lima GS, Rossi E, Wazlawick RS, Dalmarco EM. An analysis of the development of digital health technologies to fight COVID-19 in Brazil and the world. Cad Saúde Pública [Internet]. 2021 [cited 2023 Jul 20];37(3):e00243220. Available from: https://www.scielo.br/j/csp/a/rvdKVpTJq8PqTk5MgTYTz3x/?lang=pt
  » https://www.scielo.br/j/csp/a/rvdKVpTJq8PqTk5MgTYTz3x/?lang=pt

NOTES

• FUNDING INFORMATION

  This study was financed by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brazil (CAPES) - Finance Code 001. Vanessa de Brito Poveda would like to thank the support of the Conselho Nacional de Desenvolvimento Científico e Tecnológico - CNPq (Process 313857/2023-7).

• TRANSLATED BY

  Leonardo Parachú

Edited by

EDITORS

Associated Editors: Gisele Cristina Manfrini, Maria Lígia Bellaguarda Editor-in-chief: Elisiane Lorenzini

Publication Dates

• Publication in this collection
  15 Apr 2024
• Date of issue
  2024

History

• Received
  26 July 2023
• Accepted
  07 Nov 2023