DIGITAL PLATFORM FOR MANAGEMENT OF SPECIAL IMMUNOBIOLOGICALS IN PRIMARY HEALTH CARE

PLATAFORMA DIGITAL PARA LA GESTIÓN DE INMUNOBIOLÓGICOS ESPECIALES EN ATENCIÓN PRIMARIA DE SALUD

ABSTRACT

Objective:

to develop and validate a digital platform for managing special immunobiologicals in Primary Health Care.

Method:

this is a methodological study, carried out in Campinas/SP, Brazil. The platform development took place between July and December 2021. Between April and October 2022, simulation and validity were carried out by professionals, applying an instrument containing statements referring to the ISO/IEC 25010 standard characteristics and sub-characteristics, judged as in agreement, disagreement or does not apply. Values of characteristics and subcharacteristics were estimated, considering the minimum value of 70% of positive answers to be adequate. Characteristics and subcharacteristics that did not reach the minimum value had their justifications categorized according to changes made.

Results:

the platform was called ConectAPS-CRIE. Login, Home Dashboard, Users, Units, Immunobiological, Patients, Requests, Reports and Help areas were developed. When validated by nurses, the Accessibility (66.7%) and Protection Against Error (66.7%) subcharacteristics were considered inadequate. Information technology professionals considered the Reliability characteristic (52.8%) and the Functional correctness (69.2%), Accessibility (37.5%), Error protection (40%), Integrity (59.1%) and Non-repudiation (66.7%) sub-characteristics to be inadequate. The categories of justifications reported and changed were: disappearance of patients’ record when changing it; access to the system using only a password; lack of accessibility characteristics; lack of validity of personal data; exclusion of patients with a registered request; inclusion of a request without patient registration; layout and text message adjustment in the Attach File function; request disconnected from the requesting user’s unit.

Conclusion:

the platform can facilitate the management of requests for special immunobiologicals in Primary Care and its validity has allowed it to be improved.

DESCRIPTORS:
Nursing informatics; Biomedical technology; Software; Software validation; Vaccination; Immunization programs

RESUMEN

Objetivo:

desarrollar y validar una plataforma digital para la gestión de inmunobiológicos especiales en Atención Primaria de Salud.

Métodos:

estudio metodológico, realizado en Campinas/SP, Brasil. El desarrollo de la plataforma se llevó a cabo entre julio y diciembre de 2021. Entre abril y octubre de 2022, la simulación y validación fueron realizadas por profesionales, aplicando un instrumento que contiene declaraciones referentes a las características y subcaracterísticas de la norma ISO/IEC 25010, juzgada como de acuerdo, desacuerdo o no aplicable. Se estimaron valores de características y subcaracterísticas, considerando adecuado el valor mínimo de 70% de respuestas positivas. Las características y subcaracterísticas que no alcanzaron el valor mínimo tuvieron sus justificaciones categorizadas de acuerdo con los cambios realizados.

Resultados:

la plataforma se denominó ConectAPS-CRIE. Se desarrollaron las áreas de Login, Panel de Inicio, Usuarios, Unidades, Inmunobiológicos, Pacientes, Solicitudes, Informes y Ayuda. Al ser validadas por enfermeros, las subcaracterísticas Accesibilidad (66,7%) y Protección contra errores (66,7%) fueron consideradas inadecuadas. Los profesionales de TI consideraron inadecuadas la característica Fiabilidad (52,8%) y las subcaracterísticas Corrección funcional (69,2%), Accesibilidad (37,5%), Protección contra errores (40%), Integridad (59,1%) y No repudio (66,7%). Las categorías de justificaciones reportadas y modificadas fueron: desaparición del expediente del paciente al cambiarlo; acceso al sistema utilizando únicamente una contraseña; falta de funciones de accesibilidad; falta de validación de datos personales; exclusión de pacientes con solicitud registrada; inclusión de una solicitud sin registro de paciente; ajuste de diseño y mensajes de texto en la función Adjuntar archivo; solicitud desconectada de la unidad del usuario solicitante.

Conclusión:

la plataforma puede facilitar la gestión de solicitudes de inmunobiológicos especiales en Atención Primaria y su validación ha permitido mejorarla.

DESCRIPTORES:
Informática aplicada a la enfermería; Tecnología biomédica; Programas informáticos; Validação de programas de computador; Vacunación; Programas de inmunización

RESUMO

Objetivo:

desenvolver e validar uma plataforma digital para gerenciamento de imunobiológicos especiais na Atenção Primária à Saúde.

Método:

estudo metodológico, realizado em Campinas/SP, Brasil. O desenvolvimento da plataforma ocorreu entre julho e dezembro de 2021. Entre abril e outubro de 2022 realizou-se simulação e validação pelos profissionais, aplicando-se instrumento contendo afirmativas referentes às características e subcaracterísticas da norma ISO/IEC 25010, julgadas como de acordo, desacordo ou não se aplica. Foram estimados valores das características e subcaracterísticas, considerando-se adequado o valor mínimo de 70% de respostas positivas. Características e subcaracterísticas que não alcançaram o valor mínimo tiveram suas justificativas categorizadas conforme alterações realizadas.

Resultados:

a plataforma foi denominada ConectAPS-CRIE. Foram desenvolvidas áreas de Login, Painel Inicial, Usuários, Unidades, Imunobiológicos, Pacientes, Solicitações, Relatórios e Ajuda. Na validação por enfermeiros, consideraram-se inadequadas as subcaracterísticas Acessibilidade (66,7%) e Proteção contra erro (66,7%). Profissionais da área de informática consideraram inadequada a característica Confiabilidade (52,8%) e as subcaracterísticas Correção funcional (69,2%), Acessibilidade (37,5%), Proteção contra erro (40%), Integridade (59,1%) e Não repúdio (66,7%). As categorias das justificativas relatadas e alteradas foram: desaparecimento do cadastro do paciente ao alterá-lo; acesso ao sistema utilizando somente senha; ausência de recursos de acessibilidade; ausência de validação de dados pessoais; exclusão de paciente com solicitação cadastrada; inclusão de solicitação sem cadastro de paciente; ajuste de layout e mensagem de texto na função Anexar Arquivo; solicitação desvinculada da unidade do usuário solicitante.

Conclusão:

a plataforma poderá facilitar o gerenciamento de solicitações de imunobiológicos especiais na Atenção Primária e sua validação permitiu aprimorá-la.

DESCRITORES:
Informática em enfermagem; Tecnologia em saúde; Software; Validação de programas de computador; Vacinação; Programas de imunização

INTRODUCTION

Vaccination par excellence constitutes an essential action in the fight against emerging and re-emerging infections, acting to strengthen public and individual health as it is highly effective and safe, accessible for the most part and hardly affected by the development of antimicrobial resistance1^-2.

In fact, faced with a situation as relevant as this, the World Health Assembly endorsed a strategy to face the challenges of immunization between 2021 and 2030, with the objectives of maintaining the gains achieved in vaccination, recovering the losses caused by the COVID-19 pandemic and ensuring everyone receives the necessary vaccines1.

Currently, Brazil offers its population all the vaccines recommended by the World Health Organization (WHO), in accordance with the Brazilian National Vaccination Calendar (NCV), and regarding vaccination coverage in special populations, Brazil is among the top three countries in Latin America3^-4.

It should be noted that special populations are groups of individuals who have increased susceptibility to diseases or are at risk of complications for themselves or others. And there is certainly an indication that they receive immunobiological with high financial costs, provided and applied by the Reference Centers for Special immunobiologicals (CRIE - Centros de Referência para Imunobiológicos Especiais)3^,5.

A study carried out in Rio de Janeiro, Brazil, showed that expanding access to special immunobiologicals in the national territory is possible through changes in logistical processes so that these vaccines are sent on demand to Primary Health Care (PHC)6.

In the context of PHC, the participation of health professionals in immunization action organization becomes essential, as they are responsible, in addition to complying with NCV3.

In vaccination activities, the role of professional nurses stands out, who will work together with managers who acquire skills that allow using and developing health technologies3^,7^-8.

Indeed, to improve quality, surveillance and access to data from immunization programs, it is relevant to employ innovations that consider the experiences of other sectors1 such as Information and Communication Technologies (ICT), which are technological tools for communication and access to information, since these technologies contribute to organization and improvement in PHC nurses’ work process and also to increase the speed of the flow of information and, therefore, also act in interaction with other teams9.

Considering the search for strategies aimed at ensuring that everyone receives the vaccines they need, whose main focus is the immunization of the population with special clinical conditions and certainly the use of ICT as facilitating tools, this study proposes to develop and validate a digital platform for managing special immunobiologicals in PHC with the purpose of contributing to the organization of orders and particularly increasing vaccination coverage for this population.

METHOD

This is a methodological study, carried out in two stages focusing on the development and validity of a digital platform for managing special immunobiologicals in PHC, carried out in the city of Campinas/SP, Brazil.

The first stage of research was carried out from July to December 2021, and was characterized by the system development, where the digital platform software was created by a computer professional hired for this purpose. The platform content was defined by the authors and was based on the CRIE Manual 5, with its structure consisting of an environment for user registration and requests for special immunobiologicals.

The software was developed following the principles of communication (definition of content, structure and functions), planning (detailing the plan, defining costs, identifying deadlines and changes), modeling (creating the model according to client definitions and at a technical level), construction (set of coding and testing tasks that finalize a software) and availability (delivery, support and feedback), described by Pressman.10 It is noteworthy that each principle did not occur at a single moment, but repeatedly, as the system development advanced10.

Among the web development tools, the following stand out: Zend Framework, Bootstrap framework, Hypertext Preprocessor (PHP), HyperText Markup Language (HTML), Javascript and Jquery, with a database in My Structured Query Language (MySQL).

The second stage took place from April to October 2022, with simulated use and platform validity carried out by nurses from the Basic Health and Epidemiological Surveillance Units of the Northwest District of Campinas/São Paulo/Brazil and by Information technology (IT) professionals.

It is noteworthy, however, that the sample was intentional and not probabilistic, following definitions of the Brazilian Association of Technical Standards (ABNT - Associação Brasileira de Normas Técnicas), Brazilian Standard (NBR - Norma Brasileira), International Organization for Standardization/International Electrotechnical Commission (ISO/IEC) 14598-6, which consider that, to obtain reliable results, at least eight evaluators are sufficient for each category11.

A total of 32 nurses were invited to participate in the research, who had been working in the Campinas municipal health network for at least a year, with basic computer knowledge, directly involved with vaccine management or vaccination room supervision.

It professionals with at least a degree in information technology, professional experience in systems analysis or software development were included. The invitation to this group was made through a search for professionals who graduated from graduate courses related to the health area by researchers, students and technicians registered in the Directory of Research Groups of the Brazilian National Council for Scientific and Technological Development (CNPq - Conselho Nacional de Desenvolvimento Científico e Tecnológico)*, using the following search terms in the research line: “nursing science and technology”, “health care technologies” and “health information and decision support systems”, and by information technology professors. An analysis of each participant’s Lattes resume was carried out before the invitation was issued. In total, 52 IT professionals were invited.

A message was sent to the guests via email requesting their participation in the research. The message had in its content:

1. Title, objective, general data and research procedures;

2. Individual username and password to access the platform;

3. Platform email address;

4. File in Portable Document Format (PDF^®) containing the case of a fictitious patient who seeks the Basic Health Unit (BHU) to request special immunobiologicals;

5. Email address of the questionnaire created in Forms^® on the Google^® website;

6. Informed Consent Form (ICF) in PDF^® format.

A 30-day period for releasing the system to users was established.

The invitation requested that a fictitious patient’s case be inserted and attached to the platform by participants, and informed that, in case of doubt, the Help icon would give access to the Tutorial on using the platform.

After professionals interacted with the platform, the digital environment was validated by completing the questionnaire that had the ICF as its initial content, and contained statements referring to ISO/IEC 2501012 standard characteristics and sub-characteristics, adapted from Sperandio (2008)13 and Oliveira (2015)14. This same standard was used in other studies to guide the software development, validity and/or assessment in health-related applications15^-20.

Nine nurses and 10 IT professionals participated in the study. It is worth highlighting that in the instrument for judging “nurses”, statements were made referring to the characteristics and sub-characteristics related to the platform’s functional performance, while the statements used in the instrument for judging IT professionals were related to the characteristics and sub-characteristics of the software’s functional performance and technical quality.

It should also be noted that the options for professionals’ judgment regarding the “statements” included: agreement, when the platform met the requirement; disagreement, when the platform did not meet the requirement; and does not apply when research participants failed to evaluate the requirement or the requirement was judged not applicable. By choosing to judge in disagreement, research participants were able to justify their answer. Positive answers, i.e., in agreement, received a score of 1, while answers in disagreement received a score of zero. “Not applicable” answers were discarded.

The characteristic value (CV) and subcharacteristic value (SCV) indices were estimated according to the formula contained in ISO/IEC 14598-611 standard, in which: 1) CV is equal to the sum of the measured values of the subcharacteristics, divided by the number of subcharacteristics; and 2) SCV is equal to the sum of positive answers to the subcharacteristics, divided by the total number of questions, minus the number of discarded questions.

The percentage values of the characteristics were obtained using a simple rule of three. The simple percentage was used to present the value of each subcharacteristic. The data generated were recorded in an Excel^® spreadsheet. According to the validity scale for characteristics and subcharacteristics, proposed in the ABNT NBR ISO/IEC 14598-6 Annex C (Informative)11 and adapted by Sperandio,13 the minimum value of 70% of positive answers was considered adequate.

The justifications for the statements referring to the characteristics and sub-characteristics that did not reach the minimum value adequate for the platform were categorized according to the corresponding changes for their correction, which were also described. Missing justifications or without significant or complete content for updating the platform were disregarded.

Nurses and IT professionals are presented with the initials “N” and “P”, respectively, and numbered consecutively according to the order in which the questionnaire was completed for their identification regarding the reasons reported.

RESULTS

Platform development

Chart 1 describes the developed platform, called ConectAPS-CRIE, its areas, users with access and functions are described.

Chart 1 -
Access areas, users with access and functions of platform areas. Campinas, SP, Brazil, 2023.

Area	Access	Functions
Login	All	Fill in username and password for access.
Home Dashboard	All registered users	Welcome message and number of requests, patients, units and registered immunobiological. Instructions on how the platform works and sidebar with icons for accessing other areas.
Users	Administrators only	List of registered users and their data, making it possible to change, delete or register new users to access the platform.
Units	Administrators only	List of registered units, making it possible to change, delete and register new units. They can be filtered by name, district, city or state.
Immunobiological	Administrators only	List of registered immunobiological, making it possible to change, delete, register new vaccines and filter them by name.
Patients	All users, restricted to the unit in which they are registered	List of registered patients, making it possible to change, delete, insert new ones and filter by patient data. When entering a new one, there is a place to fill out patients’ registration, with name, date of birth, sex, mothers’ name, color, profession, address, telephone, Brazilian Health System (SUS - Sistema Único de Saúde) card, Identification (ID) card or Birth Certificate and underlying illness or risk condition.
Requests	All users, restricted to the unit in which they are registered	List of vaccines already requested for each patient, including the requesting unit, vaccine, date of birth, age and request status. For each request there are four icons in the Actions field: Status (view of order status history), Attachments (view of the file attached to the request or possibility of attaching other files), View (issues individual report with patient and request data, being possible to print), Change (request edit) and Delete. There is the Insert New icon to add a new request.
Reports	All users, restricted to the unit in which they are registered	List of all requests, which can be filtered by BHU, immunobiological, status, patient name, date of birth, age and justification. There is the Print icon, which allows to print.
Help	All registered users	Access to the platform usage tutorial and link to the CRIE Manual.5

Figure 1 shows the Login and Home Dashboard areas of the platform.

The Login area corresponds to the platform’s opening area. Only previously registered users with a password will have access to it. After correctly filling in user data and password, access will be provided to the Home Dashboard, where they will find the icons that lead to the rest of the system areas.

Figure 1 -
Illustration of the platform’s Login and Home Dashboard areas. Campinas, SP, Brazil, 2023.

Figure 2 shows the Insert New screens for the Patients and Request areas.

These screens correspond to the registration location for adding a new patient and requesting immunobiological, respectively. For inclusion to be effective, it is necessary to fill in the mandatory data indicated on the page as well as click on the save button.

After registering a new request, it will be listed in the Immunobiological area, thus allowing to attach files, which can be medical reports, tests, adverse event reports, discharge summaries, vaccination card, medical prescriptions and other documents in digital format.

Figure 2 -
Illustration of the Insert New screens in the Patients and Requests areas of the platform. Campinas, SP, Brazil, 2023.

Platform validity

In the platform validity stage, nine nurses and 10 IT professionals participated in simulation and completed the questionnaire.

The nurses who participated in validity were female and, of these, eight (88.9%) worked in PHC and one (11.1%) in epidemiological surveillance. Regarding the highest degree, six nurses had specialization (66.7%), two had a master’s degree (22.2%) and one only had a bachelor’s degree (11.1%). The median age among this group was 36 years, with a minimum age of 28 years and a maximum of 46 years. The median job tenure was 6 years, with a minimum of two years and a maximum of 25 years.

Among IT professionals, nine were male (90%). Training varied with three professionals trained in Systems Analysis and Development (30%), three in Computer Science (30%), two in information systems (20%), one in biomedical informatics (10%) and one in data processing (10%). Of these professionals, five worked as systems analysts (50%), three as professors (30%), one was a biomedical IT specialist (10%) and one was an administrative assistant (10%). Five of these professionals worked in hospitals and five in universities. Regarding the highest degree, five had a master’s degree (50%), three had a doctoral degree (30%) and two had a specialist degree (20%). The median age was 40 years old, with a minimum age of 23 years old and a maximum age of 52 years old. The median job tenure was 12 years, with a minimum of three years and a maximum of 27 years.

The validity results by nurses and IT professionals are described in Table 1, expressed by the characteristic and subcharacteristic values. In the validity carried out by nurses, there was no value below 70% of positive answers regarding the characteristics; however, regarding Usability, two sub-characteristics obtained scores below 70%, such as Accessibility (66.7%) and Protection against error (66.7%).

Regarding the assessment carried out by IT professionals, there was a value below 70% in the Reliability characteristic (52.8%) and in its Maturity (50%), Fault tolerance (33.3%) and Recoverability (50%) sub-characteristics. The Functional correctness (69.2%), Accessibility (37.5%), Error protection (40%), Integrity (59.1%), and Non-repudiation (66.7%) subcharacteristics also obtained values lower than the expected minimum (Table 1).

In total, 35 justifications were made by judges, 28 of which were identified as adequate for changes. The justifications were grouped into categories (I-VIII), according to the specifications of what was in disagreement and suggestions for improvement by professionals, indicated in the characteristics and sub-characteristics that did not obtain the minimum value considered adequate for the platform, and are described in Table 2, together with the corresponding professionals.

Table 1 -
Platform validity by nurses (n=9) and IT professionals (n=10), expressed by characteristic and subcharacteristic values, Campinas, SP, Brazil, 2023.

Characteristic	Subcharacteristics	Nurses			IT professionals
		Answers in agreement	Total answers	*SCV (%)	Answers in agreement	Total answers	*SCV(%)
Functional adequacy	Functional integrity	17	18	94.4	15	17	88.2
	Functional correctness	24	27	88.9	18	26	69.2
	Functional fitness	9	9	100	8	9	88.9
	†CV	-	-	94.4	-	-	82.1
Reliability	Maturity	6	6	100	4	8	50.0
	Fault tolerance	2	2	100	2	6	33.3
	Recoverability	1	1	100	3	6	50.0
	Availability	5	5	100	7	9	77.8
	†CV	-	-	100	-	-	52.8
Usability	Adequacy recognition	27	27	100	37	40	92.5
	Learnability	26	26	100	24	28	85.7
	Operability	18	18	100	15	18	83.3
	Accessibility	6	9	66.7	3	8	37.5
	Error protection	4	6	66.7	4	10	40.0
	User interface Aesthetics	18	18	100	19	20	95.0
	†CV	-	-	88.9	-	-	72.3
Performance efficiency	Time	18	18	100	19	19	100
	Resources	9	9	100	10	10	100
	Capacity	16	18	88.9	19	24	79.2
	†CV	-	-	96.3	-	-	93.1
Compatibility	Interoperability	9	12	75.0	8	9	88.9
	Coexistence	5	6	83.3	5	6	83.3
	†CV	-	-	79.2	-	-	86.1
Security	Confidentiality	8	8	100	9	9	100
	Integrity	16	19	84.2	13	22	59.1
	Non-repudiation	8	9	88.9	2	3	66.7
	Accountability	11	13	84.6	8	10	80.0
	Authentication	6	6	100	6	7	85.7
	†CV	-	-	91.5	-	-	78.3
Ease of maintenance	Analyzability	-	-	-	5	6	83.3
	Modifiability	-	-	-	3	4	75.0
	Testability	-	-	-	3	4	75.0
	Modularity	-	-	-	3	4	75.0
	Reusability	-	-	-	5	6	83.3
	†CV	-	-	-	-	-	78.3
Portability	Adaptability	-	-	-	5	6	83.3
	Installability	-	-	-	5	5	100
	Ability to replace	-	-	-	3	4	75.0
	†CV	-	-	-	-	-	86.1

• * SCV: subcharacteristic value, †CV: characteristic value.

Table 2 -
Category of justification in case of disagreement with the respective characteristics and subcharacteristics reported by nurses (N) and IT professionals (P). Campinas, SP, Brazil, 2023.

Justification category	Characteristic/subcharacteristic	Professionals
I- Disappearance of patient record when changing it	Functional adequacy/functional correctness	P2, P2, P8, P7
	Reliability/ maturity	P1
	Security/integrity	P8
II- Access to the system using password only	Functional adequacy/functional correctness	P2
	Security/integrity	P8
III- Lack of accessibility resources	Usability/accessibility	E2, E8, P2, P5, P6, P9
IV- Lack of validity of personal data	Functional adequacy/functional correctness	P8, P2, P7, P6
	Usability/error protection	E2, P1, P2, P8
V- Exclusion of a patient with a registered request	Functional adequacy/functional correctness	P2
	Security/integrity	P2, P7
VI- Inclusion of request without patient registration	Usability/error protection	P7
VII- Layout and text message adjustment in the Attach File function	Functional adequacy/functional correctness	P2
VIII- Request unlinked to the requesting user’s unit	Functional adequacy/functional correctness	P2

• * https: Hyper Text Transfer Protocol Secure.

Changes to the platform were made for each category, as follows. I: locking function, to prevent the visualization of another inserted patient, was removed; II: captcha was included when logging in; III: buttons to access the pound tool, increase page zoom and text contrast were inserted; IV: ID card validity, SUS card and date of birth was inserted, with a requirement for digits in a restricted quantity, and the restriction of a single registration per document number was also applied; V: lock was inserted so as not to exclude a patient who has a request; VI: lock was inserted to prevent requests from being registered with no link to patients; VII: there was no layout correction, but after choosing the document to be attached, the file name will appear next to the Choose Files button, with the File term having been standardized in the messages; VIII: the function of automatically linking the requesting user’s unit was inserted.

DISCUSSION

During the COVID-19 pandemic, researchers concluded that integration between different services can help improve access to vaccination, using the strategy of removing special immunobiologicals from CRIE and applying them to a specialized service; in this case, in the care provided to HIV virus carriers7.

Considering the importance of facilitating user access to health supplies, the developed platform allows the request and application of immunobiological in PHC, with their provision by CRIE. This tool is aimed at nurses, professionals who point out the usefulness of this type of technology both in Brazil and in other countries21^-23.

ConectAPS-CRIE’s intention is to replace filling out requests for immunobiological on paper or electronic mail, as it is a technological resource produced specifically for this purpose. Although some nurses may express a preference for using electronic mail in relation to specific user care record interface technologies, it is not necessarily the most appropriate24.

The system produced has spaces for filling out mandatory information, which can, at first, be seen as a difficulty22 compared to filling it out on paper or electronic mail, in which information can be hidden. However, the mandatory recording of essential information for patient safety and location was applied, ensuring that the right vaccine is applied to the right patient and facilitating communication with patients, as soon as the immunobiological is available at the BHU.

Furthermore, there are other advantages in forwarding requests for special immunobiologicals via the platform compared to electronic mail, due to the possibility of maintaining records, issuing reports and sharing information among the professionals involved. The electronic mail file can be deleted or lost as it is used for other purposes by other professionals and, furthermore, does not allow management of patient records and the delivery, receipt and application of these vaccines.

In addition to technologies with a specific purpose being neglected in relation to the use of electronic mail, a study carried out in India found that, among the barriers in using technologies by health professionals is the low competence in handling them22. As a way of assisting nurses in handling the platform, the platform has a Help page, containing the usage tutorial, serving as a guide for navigating the system as well as a link that accesses the CRIE5 Manual, with information about immunobiological, its application and indications.

In addition to providing devices to contribute to the platform applicability, we sought to qualify its use by carrying out validity by nurses and IT professionals with technical and scientific knowledge on the subject, mostly graduates and with experience in their roles.

From validity, a difference was found in the results of some aspects of the software when considering the different professional categories, which can be explained by their mastery in specific areas of knowledge.

Validity carried out by nurses demonstrated adequacy in all characteristics. However, in the validity carried out by IT professionals, the Reliability characteristic was considered inadequate.

As in other studies, Reliability validity expressed the extent of reliable execution of software functionalities, both under normal conditions and in the presence of failures15^,20.

In the justifications provided regarding Reliability inadequacy, the locking function, used to prevent the visualization of another patient inserted during platform use simulation, was seen as a system failure, which was corrected.

Regarding Usability characteristic, both in validity by nurses and IT professionals, two sub-characteristics obtained scores lower than those considered adequate, namely Accessibility and Protection against error. Similarly, Accessibility obtained unsatisfactory results in a software assessment study on preventing skin lesions in newborns, suggesting the increase in font, zoom and screen splitting capabilities as well as the existence of audios and videos, in order to promote access for people with hearing, vision or speech impairments19. Following these suggestions, buttons to access the pounds tool, increase page zoom and text contrast were inserted into the platform.

In the case of the Error Protection subcharacteristic, another system was assessed with 100% agreement responses among information technology professionals and 78.2% among nurses, showing a more favorable assessment of this software in this regard compared to the present system16.

Registration data validity and locking request registration function without patients were adjustments made to remedy the justifications of inadequacy pointed out by professionals.

The Functional correctness subcharacteristic also resulted in values below expectations from IT professionals’ validity.

This subcharacteristic is contained in the Functional suitability characteristic. Similar to the present research, a study in nursing found a difference in the opinion of groups of evaluators regarding this characteristic, discussing the difficulty of its assessment by information technology professionals, justified by the lack of knowledge regarding nursing, since issues involve this topic, and nurses, on the other hand, may have a positive assessment19.

Specifically regarding the Functional correctness subcharacteristic, the notes did not observe content specific to nursing subjects, but inadequacies regarding structural errors in the system and justifications specific to the technology area, which led to corrections such as removing the lock to observe registration of other patients, inclusion of captcha when loging in, registration data validity, locking so as not to exclude a patient who has a request and introduction of the function to automatically link the user’s unit to the request.

Finally, the Integrity and Non-repudiation subcharacteristics, belonging to the Security characteristic, also presented values below expectations among IT professionals.

Research highlighted that nurse managers believe that information security and record efficiency are potentialities in using technologies and essential for services, demonstrating the importance of this aspect in ICT use and maintenance23. Systematic literature review pointed out current research trends involving medical data privacy, secure communication and user authentication, suggesting to carry out of new studies involving software development and implementation of actions that lock illegal third-party access to clinical records and health service systems25.

The justifications considered for updating the platform were only related to the Integrity subcharacteristic and presented categories similar to the Functional correctness subcharacteristic, including the same corrections made.

Based on the above, the digital platform developed is characterized by innovation in nursing, and is capable of registering patients, controlling vaccines and applications, serving as an interface between the services involved.

A limitation of this study is the fact that, during the validity stage, it was not possible to encourage communication function and update immunobiological order status between applicant and Epidemiological Surveillance and CRIE professionals, as this process occurred in a single moment, with no subsequent response via simulation so that research participants could view it.

The communication function between services and updating order status can be used after implementing the platform in real service situations and thus have its impact fully estimated by its users.

CONCLUSIONS

The developed platform is classified as a technological tool and was designed to facilitate the process of managing requests for special immunobiologicals in PHC.

Its validity based on ISO/IEC 25010 standards allowed us to find aspects to be improved. Based on the justifications for inadequacy presented by the research participants, adjustments were made in order to make the platform adequate for implementation in PHC and, thus, impact on improving the organization of requests and increasing vaccination coverage of groups of people eligible for these vaccines.

REFERENCES

• 1. World Health Organization. Immunization Agenda 2030: A global strategy to leave no one behind [Internet]. World Health Organization [cited 2020 Apr 01]. Available from: https://www.who.int/teams/immunization-vaccines-and-biologicals/strategies/ia2030
  » https://www.who.int/teams/immunization-vaccines-and-biologicals/strategies/ia2030
• 2. MacDonald N, Mohsni E, Al-Mazrou Y, Andrus JK, Arora N, Elden S, et al. Global vaccine action plan lessons learned I: Recommendations for the next decade. Vaccine [Internet]. 2020 [cited 2023 Aug 19];38(33):5364-71. Available from: https://doi.org/10.1016/j.vaccine.2020.05.003
  » https://doi.org/10.1016/j.vaccine.2020.05.003
• 3. Souza PA, Gandra B, Chaves ACC. Experiências sobre imunização e o papel da Atenção Primária à Saúde. APS [Internet]. 2020 [cited 2023 Aug 19];2(3):267-71. Available from: https://doi.org/10.14295/aps.v2i3.57
  » https://doi.org/10.14295/aps.v2i3.57
• 4. Rombini MF, Mauas RP, Urueña A. Ranking de los programas de inmunización en América Latina, 2019. Rev Panam Salud Publica [Internet]. 2022 [cited 2023 Aug 20];46:e204. Available from: https://doi.org/10.26633/RPSP.2022.204
  » https://doi.org/10.26633/RPSP.2022.204
• 5. Brasil. Ministério da Saúde. Secretaria de Vigilância em Saúde. Departamento de Imunização e Doenças Transmissíveis. Manual dos centros de referência para imunobiológicos especiais [Internet]. Brasília: Ministério da Saúde; 2019[cited 2023 Aug 19]. Available from: https://bvsms.saude.gov.br/bvs/publicacoes/manual_centros_imunobiologicos_especiais_5ed.pdf
  » https://bvsms.saude.gov.br/bvs/publicacoes/manual_centros_imunobiologicos_especiais_5ed.pdf
• 6. Brandão LGP, Costa MD, Martins PS, Jesus-Junior SCA, Aguiar DF, Lemos AS, et al. Telemedicine in the National Immunization Program (Brazil): A promising tool. Vaccine: X [Internet]. 2022 [cited 2023 Aug 20];11:100188. Available from: https://doi.org/10.1016/j.jvacx.2022.100188
  » https://doi.org/10.1016/j.jvacx.2022.100188
• 7. Parmejani PSS, Picone CM, Alves APPS, Sartori AMC, Ibrahim KY. Facilitating access to pneumococcal vaccine for people living with HIV: An experience report. Rev Esc Enferm USP [Internet]. 2022 [cited 2023 Jun 23];56:e20210563. Available from: https://doi.org/10.1590/1980-220X-REEUSP-2021-0563en
  » https://doi.org/10.1590/1980-220X-REEUSP-2021-0563en
• 8. Mello EF, Tibério BA, Reichembach MT, Pontes L. Development of a nursing website for critical care regarding healthcare-associated infections. Rev Bras Enferm [Internet]. 2021 [cited 2023 Aug 10];74(Suppl 5):e20200928. Available from: https://doi.org/10.1590/0034-7167-2020-0928
  » https://doi.org/10.1590/0034-7167-2020-0928
• 9. Mota DN, Torres RAM, Guimarães JMX, Marinho MNASB, Araújo AF. Tecnologias da informação e comunicação: influências no trabalho da estratégia Saúde da Família. J Health Inform [Internet]. 2018 [cited 2023 Aug 19];10(2):45-9. Available from: https://jhi.sbis.org.br/index.php/jhi-sbis/article/view/563
  » https://jhi.sbis.org.br/index.php/jhi-sbis/article/view/563
• 10. Pressman RS, Maxim BR. Engenharia de Software: Uma abordagem profissional. 8th ed. Porto Alegre: AMGH; 2016.
• 11. Associação Brasileira de Normas Técnicas. NBR ISO/IEC 14598-6:2004: Engenharia de software: avaliação de produto. Parte 6: documentação de módulos de avaliação [Internet]. Rio de Janeiro; 2004[cited 2023 Jun 17]. Available from: https://www.normas.com.br/autorizar/visualizacao-nbr/23983/identificar/visitante
  » https://www.normas.com.br/autorizar/visualizacao-nbr/23983/identificar/visitante
• 12. ISO. Online Browsing Platform (OBP). ISO/IEC 25010:2011 Systems and software engineering. Systems and software quality requirements and evaluation (SQuaRE). System and software quality models [Internet]. Switzerland; 2011 [cited 2023 Jun 10]. Available from: https://www.iso.org/obp/ui/en/#iso:std:iso-iec:25010:ed-1:v1:en
  » https://www.iso.org/obp/ui/en/#iso:std:iso-iec:25010:ed-1:v1:en
• 13. Sperandio DJ. A tecnologia computacional móvel na sistematização da assistência de enfermagem: Avaliação de um software-protótipo [dissertation]. Ribeirão Preto: Universidade de São Paulo, Escola de Enfermagem de Ribeirão Preto; 2008 [cited 2023 Aug 19]. 141 p. Available from: https://www.teses.usp.br/teses/disponiveis/22/22132/tde-11092008-165036/publico/DirceleneJussaraSperandio.pdf
  » https://www.teses.usp.br/teses/disponiveis/22/22132/tde-11092008-165036/publico/DirceleneJussaraSperandio.pdf
• 14. Oliveira NB, Peres HHC. Evaluation of the functional performance and technical quality of an electronic documentation system of the nursing process. Rev Latino-Am Enfermagem [Internet]. 2015 [cited 2023 Aug 10];23(2):242-9. Available from: https://doi.org/10.1590/0104-1169.3562.2548
  » https://doi.org/10.1590/0104-1169.3562.2548
• 15. Vernier LS, Dartora J, Herbert J, Cazella SC, Levandowski DC. Model for quality analysis of neonatal hearing screening software: Theory applied. Int J Med Inform [Internet]. 2021 [cited 2023 Aug 22];150:104435. Available from: https://doi.org/10.1016/j.ijmedinf.2021.104435
  » https://doi.org/10.1016/j.ijmedinf.2021.104435
• 16. Medeiros RM, Santos MN, Moraes VM, Kaiser DE, Duarte ERM, Viegas K. Validação e aplicabilidade do software Wounds Monitoring na avaliação e monitoramento de feridas. Glob Acad Nurs [Internet]. 2022 [cited 2023 Aug 22];3(2):e249. Available from: https://doi.org/10.5935/2675-5602.20200249
  » https://doi.org/10.5935/2675-5602.20200249
• 17. Barletta VS, Caivano D, Colizzi L, Dimauro G, Piattini M. Clinical-chatbot AHP evaluation based on “quality in use” of ISO/ IEC 25010. Int J Med Inform [Internet]. 2023 [cited 2023 Aug 10];170:104951. Available from: https://doi.org/10.1016/j.ijmedinf.2022.104951
  » https://doi.org/10.1016/j.ijmedinf.2022.104951
• 18. Felipe GF, Lima FET, Barbosa LP, Moreira TMM, Joventino ES, Freire VS, et al. Evaluation of user embracement software with pediatric risk classification. Rev Bras Enferm [Internet]. 2020 [cited 2023 Aug 11];73(3):e20180677. Available from: http://doi.org/10.1590/0034-7167-2018-0677
  » http://doi.org/10.1590/0034-7167-2018-0677
• 19. Santos SV, Ramos FRS, Costa R, Batalha LMC. Assessment of the quality of a software application for the prevention of skin lesions in newborns. Rev Latino-Am Enfermagem [Internet]. 2020 [cited 2023 Aug 10];28:e3352. Available from: http://doi.org/10.1590/1518-8345.3711.3352
  » http://doi.org/10.1590/1518-8345.3711.3352
• 20. Melo WS, Sousa IS, Mariano SPS, Barbosa AS, Feitosa DSLL, Freire VECS, et al. Wise Infant Development®: Creation of a software for teaching in pediatric nursing education. Rev Bras Enferm [Internet]. 2022 [cited 2023 Aug 14];75(5):e20210466. Available from: https://doi.org/10.1590/0034-7167-2021-0466
  » https://doi.org/10.1590/0034-7167-2021-0466
• 21. Fernandes BCG, Silva Júnior JNB, Guedes HCS, Macedo DBG, Nogueira MF, Barrêto AJR. Use of technologies by nurses in the management of primary health care. Rev Gaúcha Enferm [Internet]. 2021 [cited em 2023 Aug 14];42(esp):e20200197. Available from: https://doi.org/10.1590/1983-1447.2021.20200197
  » https://doi.org/10.1590/1983-1447.2021.20200197
• 22. Faujdar DS, Singh T, Kaur M, Sahay S, Kumar R. Stakeholders’ perceptions of the implementation of a patient-centric digital health application for primary healthcare in India. Healthc Inform Res [Internet]. 2021 [cited 2023 Aug 14];27(4):315-24. Available from: https://doi.org/10.4258/hir.2021.27.4.315
  » https://doi.org/10.4258/hir.2021.27.4.315
• 23. Vandresen L, Pires DEP, Martins MMFPS, Forte ECN, Leão E, Mendes M. Potentialities and difficulties of technological mediation in the work of nurse managers in hospitals. Texto Contexto Enferm [Internet]. 2022 [cited 2023 Jun 06];31:e20220173. Available from: https://doi.org/10.1590/1980-265X-TCE-2022-0173en
  » https://doi.org/10.1590/1980-265X-TCE-2022-0173en
• 24. Ribeiro OM, Martins MM, Vandresen L, Silva JM, Cardoso MF. Usefulness of information and communication technologies: portuguese nurses’ look. Texto Contexto Enferm [Internet]. 2021 [cited 2023 Jun 06];30:e20190139. Available from: https://doi.org/10.1590/1980-265X-TCE-2019-0139
  » https://doi.org/10.1590/1980-265X-TCE-2019-0139
• 25. Marques ICP, Ferreira JJM. Digital transformation in the area of health: Systematic review of 45 years of evolution. Health Technol [Internet]. 2020 [cited 2023 Aug 19];10:575-586. Available from: https://doi.org/10.1007/s12553-019-00402-8
  » https://doi.org/10.1007/s12553-019-00402-8

NOTES

• ORIGIN OF THE ARTICLE

  This study is part of a thesis - Desenvolvimento, validação e implantação de uma Plataforma Digital para o gerenciamento de imunobiológicos especiais na Atenção Primária à Saúde”, presented to the Professional Doctoral Graduate Program at the Universidade Estadual Paulista Júlio de Mesquita Filho in 2025.

• FUNDING INFORMATION

  The present study was carried out with the support of the Coordination for the Improvement of Higher Education Personnel - Brazil (CAPES - Coordenação de Aperfeiçoamento de Pessoal de Nível Superior). Financing Code 001.

• APPROVAL OF ETHICS COMMITTEE IN RESEARCH

  Approved by the Ethics Committee in Research of the Faculdade de Medicina de Botucatu / Universidade Estadual Paulista "Júlio de Mesquita Filho", Opinion 5.019.782/2021, CAAE (Certificado de Apresentação para Apreciação Ética - Certificate of Presentation for Ethical Consideration) 51835021.1.0000.5411.

Edited by

EDITORS

Associated Editors: José Luís Guedes dos Santos, Ana Izabel Jatobá de Souza. Editor-in-chief: Elisiane Lorenzini.

Publication Dates

• Publication in this collection
  22 Dec 2023
• Date of issue
  2023

History

• Received
  31 Aug 2023
• Accepted
  10 Oct 2023