Evaluation of business process management systems (BPMS): analysis of Bizagi and Bonita softwares

ABSTRACT

Introduction:

This article presents the results of a research work that evaluated the operation of the BPMS Bizagi and Bonita, used to manage the process of hiring access services to digital books databases, conducted by the Central Library of the Federal University of Paraíba.

Objective:

The article had as goal to analyze and compare two softwares used for mapping the informational flow in business processes - the Business Process Management Systems (BPMS) Bizagi and Bonita.

Methodology:

The softwares were used for mapping and analyzing the process flow of contracting services for access to digital book databases, conducted by the Central Library of the Federal University of Paraíba. They were evaluated based on the recommendations of the Analytic Hierarchy Process AHP, considering seven criteria: license type, simulation tool, user support, usability, languages, training and documentation. The methodology adopted in this article is classified as descriptive, with a mixed quanti/qualitative approach.

Results:

As a result, it was possible to evaluate and verify, through the techniques of AHP, the functioning of Bizagi and Bonita as tools for mapping the information flow in the presented business process, observing their performances.

Conclusion:

At the end of the research, it was concluded that Bizagi was the most appropriate BPMS for the selected criteria.

KEYWORDS
Information flows; Management processes; Information management; Multi-criteria decision analysis; Analytic Hierarchy Process.

RESUMO

Introdução:

Este artigo descreve o resultado de trabalho de pesquisa que avaliou o funcionamento dos BPMS Bizagi e Bonita, utilizados para gerenciar o processo de contratação de serviços de acesso a bases de dados de livros digitais, conduzido pela Biblioteca Central da Universidade Federal da Paraíba.

Objetivo:

O artigo teve como objetivo analisar e comparar dois softwares utilizados para mapeamento do fluxo informacional em processos de negócios - os Business Process Management Systems (BPMS) Bizagi e Bonita.

Metodologia:

Os softwares foram utilizados para mapeamento e análise do fluxo do processo de contratação de serviços de acesso a bases de dados de livros digitais, conduzido pela Biblioteca Central da Universidade Federal da Paraíba. Foram avaliados com base nas recomendações do Analytic Hierarchy Process AHP, observando sete critérios: tipo da licença, ferramenta de simulação, suporte ao usuário, usabilidade, idiomas, treinamento e documentação. A metodologia adotada neste artigo é classificada como descritiva, de abordagem mista quanti/qualitativa.

Resultados:

Como resultado, foi possível avaliar e verificar, por meio das técnicas do AHP, o funcionamento do Bizagi e do Bonita como ferramentas de mapeamento de fluxo informacional no processo de negócios apresentado, observando seus desempenhos.

Conclusão:

Ao final da pesquisa, verificou-se que o software Bizagi se apresentou como o BPMS mais adequado diante dos critérios selecionados.

PALAVRAS-CHAVE
Fluxos da informação; Processos de gestão; Gestão da informação; Métodos multicritério de análise de decisão; Analytic Hierarchy Process.

1 INTRODUCTION

The development of society and the constant changes and innovations that are present in the contemporary world cause great impacts on the way individuals conduct their activities. Current demands for speed, instantaneity, adaptability and efficiency fall not only on people individually, but also exert a strong influence on organizations, which have come to exist and perform their tasks in increasingly dynamic and complex environments.

Such complexity requires organizations to systematize themselves in order to materialize their functions and activities through structures called processes, defined as “an aggregation of activities and behaviors performed by humans or machines to achieve one or more results” (ABPMP, 2013, p. 35).

As a way to manage these processes so that they are executed to the fullest of their potential, extracting the best possible results from them, some strategies can be used, such as Business Process Management (BPM) or Business Process Management. For Dumas et al. (2018, p. 6), BPM is conceptualized as “a set of methods, techniques and tools to identify, discover, analyze, redesign, execute and monitor business processes in order to optimize their performance” (author’s translation).

In this line, it is noticeable that BPM aims to analyze processes, providing an opportunity to know their operational or informational flow, components and participants in detail, and then promote systemic and continuous improvements, aiming to achieve maximum efficiency. According to Rios (2019, p. 25)

Information flows are implemented through the stages of creation/acquisition, display, storage, retrieval, sharing and/or use of information, which directly contributes to the achievement of institutional objectives, as well as supporting decision-making.

To this end, this theoretical field makes use of advances and developments in information and communication technologies, such as Business Process Management Systems (BPMS).

BPMS are software capable of executing and handling business processes through their diagrams, facilitating the visualization of information flows, allowing analysis and promotion of improvements, the simulation of processes and their eventual automation (WESKE, VAN DER AALST and VERBEEK, 2004). However, given the many BPMS options available on the market, organizations may have difficulties in selecting the most appropriate software to manage their business processes. Thus, it is essential to establish clear criteria and submit these criteria to decision-making guided by specific techniques that make it possible to make the best possible choice, enabling the selection of the BPMS that is most compatible with your business processes and objectives. strategic goals that will be achieved through them.

In this sense, the application of Multicriteria Decision Analysis Methods (MMAD) has the potential to help and support the choice process, as these methods lead to an evaluation of multiple criteria in order to provide a decision-making process based on an examination. depth of the attributes of each alternative.

Starting, then, from the practical application of BPMS tools for the management of the process responsible for contracting collections of digital book databases at the Central Library of the Federal University of Paraíba, the objective of this article emerged, proposing to use one of the most traditional and referenced MMADs in the literature, the Analytic Hierarchy Process (AHP), to promote the analysis and, consequently, the choice of a BPMS between two of the most cited and used in the market: Bizagi and Bonita.

2 THEORETICAL PATH TAKEN FOR THE DEVELOPMENT OF ANALYSIS

Making choices is a customary action for each and every individual, since even the simplest and most everyday activities involve a decision. Thus, it is natural to conclude that decision-making is also an inseparable part of organizational routines, being essential for the accomplishment of the most diverse tasks, the execution of plans, the establishment of goals and even for the definition of its strategic objectives.

Decision making, which can be conceptualized as “the process of identifying a problem or opportunity and selecting a course of action to solve it” (LACHTERMACHER, 2016, p. 3), is presented in an incessant way in the day-to-day of any organization, being applicable with a level of greater or lesser complexity depending on the organizational aspect that is subject to that particular choice.

The particularities inherent to modern business environments imply an accentuation in the complexity of the demands directed to organizations, which are constantly pressured to reach very high levels in order to guarantee their subsistence in a competitive environment in which only the most apt entities are successful. As a consequence, the processes that achieve the business objectives of these organizations also become more complex, requiring the use of techniques such as business process management to make them more efficient. From this perspective, the application of business process management also involves decisions of the most diverse types that must be taken by the actors that are part of this management structure.

Given this reality, decision-making must be recognized as a methodical and necessary process. In practice, however, it is not always performed in a technical and previously thought out way, being sometimes relegated to an automated and intuitive action, labels that do not match the true structure that constitutes this activity. Decision Theory then emerges as a way to support the decision-making process, enabling the decision-maker to reach the best possible solution for the decision problem that presents itself before them.

Still on this track, it is worth mentioning that the growing complexity of the problems faced by organizations implies more difficult decisions to be made. The obstacles to the decision-making process become even more evident as the deepening of organizational dynamics leads to the emergence of an increasing number of criteria to be properly appreciated.

Along these lines, according to Yu (2011), it is inevitable that a greater number of variables present in a decision-making process causes an increase in its level of difficulty. This understanding is also shared by Choo (2003, p. 324), who states that “those who are responsible for decisions face great complexity and uncertainty when trying to understand what the problems are, identify possible alternatives, calculate probable outcomes and clarify and order preferences”.

Thus, in order to give the deserved prominence to an activity of such significance, several methods and techniques were developed that make it more grounded, bringing more reliability to the decision that is eventually taken. Among the methods created for this purpose, the so-called Multi-Criteria Decision Analysis (MCDA) are a highlight here, the features of which will be presented below.

2.1 General aspects about the Multicriteria Decision Analysis Methods

The Multicriteria Decision Analysis Methods (MMAD) found roots for their development, initially in the period of the Second World War, when Operations Research was widely disseminated and used by the British military - and later by the US - as an analytic tool of strategic management and problem solving found in the war context (BONINI et al., 2016). In the post-war period, the experience acquired through the use of Operations Research was extended to several other areas, providing the opportunity to solve problems through the creation of mathematical models and algorithms.

Later, in the late 1960s and early 1970s, scientific studies began to emerge aiming to fill gaps originating from methodologies based on Operations Research, which proved to be excessively quantitative and incapable of adequately covering some specific classes of decisions that involve multiple and impossible criteria to be mathematically quantified. It is in this context, then, that the MMAD arise, which, although they also involve mathematical principles, are based on a subjective bias, insofar as they take into account the individual perceptions of those who are part of the decision-making process (YU, 2011).

Thus, the MMAD emerged as a tool capable of supporting the decision-making process, enabling the decision-maker to analyze different scenarios and aspects and leading to more assertive and well-founded choices. This does not mean, however, that the decision taken after applying the methods in question will necessarily be unquestionable and immutable, since the environment in which the choice is made is also oscillating.

That said, the MMAD should be seen not as a solution or a definitive guide for decision making, capable of removing all traces of subjectivity and difficulty inherent to this task, but rather as a means to help the decision maker to arrive at a well-educated final choice (LEHNHART, 2016).

Therefore, the aforementioned class of methods proposes the analysis of the various criteria presented as relevant or desired for decision making, assigning a specific valuation to each one, according to the judgment of the individual (or group) responsible for the decision-making process. In the end, it is intended to obtain a decision that allows the achievement of the pre-established objective, which is done through the correct appreciation of the preferences and value judgments attributed to the criteria pertinent to the choice process.

Initially, for a better understanding of the application of MMAD, it is necessary to consider that the decision-making process can be quite complex, being subject to the influence of a series of variables. Indeed, for Lachtermacher (2016), it is possible to list some factors capable of affecting decision-making, such as: time; the level of importance of the decision; the environment where it will be taken (which, in the case of organizations, also encompasses the influence of their culture); the degree of certainty, uncertainty and risk involved in the decision making; decision-making agents; and, finally, possible conflicts of interest.

In this vein, it is understood that the Multicriteria Decision Analysis Methods have their best application when dealing with decision problems that are primarily affected at the subjective level, that is, at the decision maker's level, being used when the existence of multiple criteria can make the choice difficult.

It is, therefore, based on the decision-maker's preferences that the various criteria selected as a parameter for the examination of alternatives will be hierarchically classified. Thus, it is possible to reach the most adequate solution for the problem situation that gave rise to the need to start the decision-making process.

In fact, as stated by Mateo (2012, p. 7), MMADs are suitable “to deal with complex problems that present high uncertainty, conflicting objectives, different formats of data and information, multiple interests and perspectives, and to explain complex and evolving biophysical and socioeconomic systems”.

Having clarified the context in which the application of Multicriteria Decision Analysis Methods is seen as conducive to contributing to the decision-making process, extracting the greatest possible benefit from this category of methods, it is appropriate to make a reservation that relates to the plurality of approaches that are categorized as MMAD.

It is worth noting the fact that not all types of MMAD can be applied to the same category of problem, and there is no universal or sufficiently versatile method to be used indistinctly in any and all situations (LEHNHART, 2016). Because of this, it is crucial for a qualified decision making that the decision maker selects the most appropriate approach from the beginning, as a wrong choice can lead to a final solution that is unable to satisfactorily address the problem raised.

From the structuring of the problem - an essential step for decision making -, it is possible to establish the bases for a better understanding of the context that surrounds it, which also makes it possible to choose the appropriate method. Once the decision-making problem to be solved has been defined and the desired objectives with the decision-making have been established, the execution of the MMAD must be carried out in compliance with an appropriate sequence so that, at the end of the decision-making process, it is possible to reach the decision that is shown to be the most compatible with the criteria and preferences established by the decision maker.

Regarding the types of MMAD, it is common to find in the literature a classification for this category of methods according to three different approaches, according to the classification given by Roy (1996), namely: single-criteria synthesis approach, subordination approach and interactive.

The AHP method, used for the development of this article, fits into the single-criteria approach of synthesis, which implies a comparative analysis of the alternatives according to their performance in the aggregation phase. Thus, through this numerical approach, the criteria are seen as a utility function, whose order of preference previously established allows the decision maker to see how a given action is capable of providing the value expected by him. This approach is based on an additive criterion, which makes possible a kind of compensation between excellent and bad performances. Along these lines, an alternative that performs poorly on some criteria may have a final result offset by a good performance on other criteria (GUARNIERI, 2015).

3 METHODOLOGY

The methodological procedures adopted for the conception of this article started, initially, from a bibliographical research to facilitate the understanding on the theme of the Multicriteria Methods of Decision Analysis, as well as on the Business Process Management Systems and their application to the management of business processes and mappings of information flows. For Araújo et al. (2018) flow mapping is the representation of the path taken by information through its channels, identifying the agents responsible for creating, moving, storing and distributing this information, regardless of whether the support is physical or digital. The authors assert that regardless of the format in which the information is recorded, “the information flow will support one or several processes of the organization” (2018, p. 3039).

Then, a documental research was carried out, which allowed the verification of all the necessary nuances for a deep knowledge of the BPMS. It was noticeable, during the research, the great variety of existing tools in the market, with the most diverse functions. Some of the most used BPM software are the following:

• - Visio: Microsoft tool aimed at creating diagrams, organization charts and various flowcharts, supporting the construction of models with BPMN notation.

• - Heflo: platform that offers as main resources the documentation, publication, control and automation of business processes.

• - Bizagi: system that uses the BPMN notation and offers the modeling, simulation, automation and control of business processes through three different and complementary platforms (Modeler, Studio and Automation).

• - BPMN.io: online platform that allows the creation of diagrams with BPMN, DMN (Decision Model and Notation) and CMMN (Case Management Model and Notation) notations.

• - Modelio: an open source tool that has the differential aspect of supporting different notations, such as UML (Unified Modeling Language), BPMN, ArchiMate and SysML (Systems Modeling Language).

• - Bonita: open source platform that enables the automation and improvement of business processes, in addition to allowing the customization of the system to ensure an individualized experience adapted to each organization or individual.

• - Sydle: software developed by a Brazilian company aimed at providing opportunities for the integration of various functions of a company, such as process automation, document management, customer relationships, data analysis, among others.

For this study, two BPMS were selected, Bonita (version 2022.1) and Bizagi (version 3.9.0.015), as they are two systems of considerable relevance in the market. In this sense, Bizagi presents itself as a software that covers more than one million users and more than nineteen thousand organizations, having as clients companies such as Adidas, Takeda Pharma, Harrods and Citizens Bank (BIZAGI, 2022). Bonita, in turn, serves companies such as the United States Department of Health, the University of Massachusetts, the French Department of Defense and Xerox (BONITA, 2022).

Both are included in the Business Process Management Platforms Reviews and Ratings list presented by Gartner, which indicates the 20 best evaluated BPBMs, the choice was still based on the wide dissemination of said BPMS in organizations based in several countries around the world, the ease of downloading the platforms (that do not have different operational requirements for their handling).

It is also necessary to mention that the two software are referenced in the academic literature that deals with Business Process Management Systems. During the review process for this work, the term “Bizagi” on Google Scholar returned 9870 results. On the other hand, the set of words “Bonita BPM” presented 3360 results, both being among the most cited in this platform, which demonstrates that BPMS are objects of studies and scientific production.

In addition, the analysis of the performance of these tools was carried out considering their application in the management of a real process, corresponding to the contracting of access license services to digital book bases, carried out within the scope of the Central Library of the Federal University of Paraíba (UFPB).

This process is classified as an unenforceability of bidding, insofar as it promotes the contracting of services offered on an exclusive basis, as each content aggregator provides a specific collection. In this way, the collections presented by the suppliers are different, in the case of a single product distributed by each company.

It is a process of great relevance for the entire academic community and, indirectly, for society, since it promotes access to virtual bibliographic collections within the University, disseminating information. The virtual collection corresponds to one of the criteria used by the National Higher Education Assessment System (SINAES) for the evaluation of undergraduate courses, as well as by the Coordination for the Improvement of Higher Education Personnel (CAPES) in relation to postgraduate courses (masters and doctorates).

The process in question, owned by the UFPB Central Library, has seven participants, most of them internal to the institution, with the exception of the supplier. Thus, we have that the process is carried out in seven instances, namely: the requester, the library management, the planning team, the accounting department, the purchasing department, the Legal Department and the supplier.

Thus, by promoting the management of this process with the help of BPM tools Bizagi and Bonita, it was possible to build a diagram of the process and, later, simulations and analyzes that made it possible to propose improvements to its information flow. In addition, applying the BPM assumptions to the process of contracting digital databases allowed the practical use of the Bizagi and Bonita software, prompting the evaluation of the systems and the comparison of their performance with the multi-criteria analysis.

Regarding MMAD, the study started with the proposal to analyze and compare information flow mapping and business process simulation (BPMS) software in order to choose the one that presents the best overall performance against the listed criteria. In this line, we opted for the approach of the single synthesis criterion, through the Analytic Hierarchy Process (AHP) method, idealized by Saaty.

To select the specific method that would be used to carry out this research, the breadth of use, clarity of the method, ease of application, stability, possibility of validation and subsequent replication were taken into account. In addition, AHP presents itself as a versatile method, being used to solve decision-making problems in several areas of knowledge and by a wide range of organizations (SAATY, 2008).

Costa (2002) indicates that the AHP is based on three basic premises: construction of hierarchies, definition of priorities and logical consistency. Due to its long tradition among multicriteria decision analysis methods, the AHP reveals itself as a robust method that, until the present day, is capable of promoting a well-designed, logical and clear decision-making process, coordinating subjective (or qualitative) aspects and mathematical procedures.

Thus, the use of the AHP method allowed us to examine the selected BPMS and, in sequence, to choose the one that presented the best performance according to the established criteria.

4 RESULTS AND DISCUSSIONS

To perform the multi-criteria analysis with the AHP, a basic structure must be followed. The very name of the method (translated as Hierarchical Analytical Process) suggests that its structure involves a hierarchy of decision steps, as a way of decomposing the decision problem and facilitating its understanding. Given this, Vargas (1990) argues that the application of the AHP takes place in two stages, namely, the hierarchical structuring stage and the evaluation stage.

For hierarchical structuring, it is necessary to detail the decision problem through the identification of the following elements: objective of the decision, criteria and alternatives. On the other hand, the evaluation stage presupposes, first, the identification of the weight of the criteria, the analysis of the performance of the alternatives in relation to each criterion and, finally, the global aggregation of the results.

Regarding the hierarchical structuring, the objective of the decision and the selection of actions and alternatives have already been presented, considering that the research problem involves the decision between the BPMS Bizagi and Bonita.

Therefore, the other steps of the AHP will be presented below, starting with the next phase, which corresponds to the identification of criteria and the hierarchical structuring of the decision problem.

4.1 Survey of criteria and hierarchical structuring

One of the fundamental premises of the category of decision support methods used in this research is the existence of multiple criteria to be considered in order to reach a well-informed decision. Thus, after defining the possible alternatives that make up the decision-making problem, it is necessary to establish the criteria that will be the object of the analysis.

This is a crucial step in the method, which represents the core of the decision-making process, since the characteristics of the choice options are what distinguish them from one another, placing the decision-maker in a position of uncertainty. Thus, to complete the choice, the decision maker needs to define what are the aspects of interest, that is, what he seeks to find in the alternative that will be chosen.

Once the BPMS Bizagi and Bonita were properly handled, allowing a greater understanding of all their constitutive aspects, it was possible to establish seven criteria as guidelines for their evaluation, taking into account the necessary characteristics for a good process management. Therefore, the following criteria were established: license type, simulation tool, user support, usability, languages, training and documentation. In this way, a hierarchical structure model was produced that represents the decision problem at three levels (objective, criteria and alternatives):

Figure 1
Hierarchical structuring of the decision problem

With all the elements necessary for the decision duly established, it was possible to proceed to the evaluation phase, which included the attribution of weights to the criteria, the analysis of alternatives according to the listed criteria and, finally, the global analysis of the performance. The conduction of these steps will be presented below.

4.2 Decision-making process evaluation stage

Having concluded the definition of the criteria that were used to examine and submit the tools to the appropriate evaluations, presented in the previous subsection, the consequence is that we reach the evaluation stage foreseen in the AHP method. In fact, this phase initially encompasses the assignment of weights to the criteria; partial performance analysis; and, finally, the final aggregation of the results.

The attribution of weights to the criteria is based on the idea of Saaty (2004), creator of the AHP, who understands that the decision maker, faced with multiple criteria, will see some as more or less important than others. The definition of this level of importance is, therefore, essential for the final result of the application of the AHP to be able to safely express the best alternative.

With this, it is necessary to start from the decision-maker's value judgment to list the criteria in order of importance, which is done through the application of a series of techniques that have as a final result the arrangement of the criteria in a scale of relative importance with a good level of reliability. The techniques conceived by Saaty were implemented as a way of guaranteeing greater precision in the ordering of criteria, avoiding inaccuracies.

The first step to calculate the relative priority of the criteria is to perform a pairwise comparison between them, using the Saaty Fundamental Scale as a reference, which will be presented in the table below:

Chart 1
Fundamental scale of absolute numbers

Intensity of importance	Definition	Explanation
1	Equal importance	Both activities contribute equally to the objective.
3	Moderate importance	Experience and judgment slightly favor one activity over the other.
5	Strong importance	Experience and judgment strongly favor one activity over another.
7	Very strong or demonstrated importance	One activity is very strongly favored over the other; its dominance is demonstrated in practice.
9	Extreme importance	The evidence favoring one activity over the other is of the highest possible order of affirmation.
2, 4, 6, 8	Intermediate values	Used when the decision maker feels the need to make concessions between the immediately lower value and the immediately higher value.
1/2, 1/3, 1/4, 1/5, 1/6, 1/7, 1/8, 1/9	Reciprocal	Used due to the principle of reciprocity that underlies the AHP.

Saaty (2004)

From this scale, it is possible to assign a numerical rating according to the level of importance given to each criterion in relation to another, by comparing the pairs of criteria. Consequently, in the end, all criteria will have been compared with each other, receiving a score (higher or lower) according to the level of importance assigned in that comparison sequence.

Thus, based on the scale above, a matrix was created in which the relative priorities of the criteria were expressed, shown in the following table:

Table 1
Criteria comparison matrix

Choice of BPMS	License Type	Simulation tool	User support	Usability	Language	Training	Documentation
License type	1	1/5	1/3	1/6	3	1/5	1/3
Simulation tool	5	1	4	1/3	6	1/2	3
User support	33	1/4	1	1/5	4	1/4	1/2
Usability	6	3	5	1	7	5	4
Language	1/3	1/6	¼	1/4	1	1/6	1/4
Training	5	2	1/5	1/5	6	1	3
Documentation	3	1/3	¼	1/4	4	1/3	1

Source: Elaborated by the authors (2022)

However, in order to obtain the priority vector (that is, the weight value) of each of the criteria, it was necessary to submit the matrix presented in the table above to some operations, the first being its normalization, whose calculations were conducted with the aid of Microsoft Excel software. For the normalization procedure, Moraes and Santaliestra (2008) indicate the need to follow the following steps:

• - First step: obtaining a total value, through the sum of each column presented in the comparison matrix.

• - Second step: division of each of the values of the matrix cells by the value corresponding to the sum of each column, as obtained in step 1. The result obtained after performing the operations referring to this step can be represented by the following table:

Table 2
Second step of the comparison matrix normalization process

Choice of BPMS	License Type	Simulation tool	User support	Usability	Language	Training	Documentation
License type	0,043	0,029	0,020	0,064	0,097	0,045	0,028
Simulation tool	0,214	0,144	0,241	0,129	0,194	0,112	0,248
User support	0,129	0,036	0,060	0,077	0,129	0,056	0,041
Usability	0,257	0,432	0,302	0,386	0,226	0,449	0,331
Language	0,014	0,024	0,015	0,055	0,032	0,037	0,021
Training	0,214	0,288	0,241	0,193	0,194	0,225	0,248
Documentation	0,129	0,048	0,121	0,096	0,129	0,075	0,083

Source: Elaborated by the authors (2022)

Once the table was normalized, a calculation of the arithmetic average of the values present in each of its lines was performed, allowing to obtain the weight of the criteria according to the table below:

Table 3
Weight assigned to criteria

Criteria	Weight
Usability	0,340
Training	0,229
Simulation tool	0,183
Documentation	0,097
User support	0,076
License type	0,046
Language	0,028

Source: Elaborated by the authors (2022)

After performing pair comparisons and obtaining the values of priorities, the techniques of the AHP method give rise to the need to verify whether the judgments made were consistent. Saaty (1990) understands that inconsistency is inherent to human nature, so that a certain degree of inconsistency must be tolerated. Therefore, he judges acceptable a maximum degree of inconsistency in the judgments corresponding to 10% (or 0.1). As a result, it was necessary to submit the judgment matrix constructed here to some calculations in order to verify if its level of consistency is within the appropriate limits, as proposed by Saaty.

Initially, we must keep in mind that the calculation of the Consistency Ratio (CR) is given by the formula CR = CI/RI, with CI corresponding to the Consistency Index and RI corresponding to the Random Index. In this sense, we have that the RI is a value proposed by Saaty (2004) that varies according to the order (n) of the matrix, which can be represented by table 5:

Table 4
Random indices calculated by Saaty

N	1	2	3	4	5	6	7	8	9	10
IR	0	0	0,52	0,89	1,11	1,25	1,35	1,40	1,45	1,49

Saaty (2004)

Therefore, it appears that for a judgment matrix of order 7 - in the case of the one produced in this research - the RI is 1.35. Thus, to calculate the Consistency Ratio, it remains only to find the value of the CI, which is obtained through the following equation:

$$\mathrm{I}\mathrm{C}=\frac{{\lambda }_{\mathrm{m}\mathrm{a}́x}-n}{n-1}$$

For the equation above, λmax corresponds to the maximum eigenvalue of the matrix, which was found through some operations (also carried out with the aid of Microsoft Excel), whose execution is well explained by Costa (2002, p. 71). First, the non-normalized values of the judgment matrix were used, and these values were duly multiplied by the priority vectors (weights) obtained for each criterion. Then, the values of each of the lines were summed in order to obtain a total value. Finally, the values resulting from this sum were divided by the priority vector of each of the criteria.

With this, it became possible to obtain the value of λmax (ie, the maximum eigenvalue of the matrix), which corresponds to the average of the values resulting from the last operation, that is, 7.423. Once the value of λmax was known, we were able to calculate that the CI (consistency index) according to the equation presented above, is 0.07.

Thus, considering that the calculation of the Consistency Ratio (RC) is given by dividing the value of CI (0.07, obtained through the operations described above) and the value of IR (1.35, calculated by Saaty as a function of of order 7 of the matrix), we have that the RC of our priority matrix is 0.07/1.35, totaling 0.052 (or 5.2%). This data indicates, therefore, that the matrix of priorities elaborated is consistent within the limits idealized by Saaty, who admitted a maximum CR of 10% as acceptable.

After these calculations, it was possible to confirm the accuracy of the values of the relative priorities and use them as weights for the criteria, which made it possible to analyze the performance of the alternatives in the light of the defined criteria.

At this stage, the criteria and characteristics established in the previous stage will serve as a benchmark for examining the BPMS Bizagi and Bonita. Based on the AHP techniques, the performance analysis of the alternatives against the criteria leads to the attribution of valuation judgments that must be expressed in a judgment matrix, according to the indices contained in the Fundamental Saaty Scale. In other words, the AHP method implies that the judge converts the results of the performance analysis into judgment matrices.

Thus, the seven matrices that translate judgments into numbers will be presented here, as indicated in the Saaty scale. Below, the results obtained in the analysis of the software will be presented according to the desired and expected characteristics in relation to the criteria defined as an evaluation parameter.

a) First criterion: license type

For this criterion, the analysis took into account the benefits that each type of license can offer the user of BPMS. In fact, the best performance in this criterion is achieved when the software offers a free license (freeware) and of the free or open source type. On the other hand, the worst performance is presented with a combination of licenses that offer more restrictions to the user, that is, the paid license and the proprietary license.

Regarding the free license, we have that it is a desired criterion because it allows the user to use the product without the expense of financial resources. For organizations, for example, it is advantageous to offer free software aimed at managing processes, given that this function is often seen as a middle function in the organization, which can prevent greater investments in appropriate tools.

As for whether the license is of the free type or of the proprietary type, it is understood that the first is more advantageous to the user as it ensures a level of freedom of unrestricted use. According to the definition presented by Stallman (2009), free software can be used according to the user's desire, who from accessing their source code can adapt it to their needs, study and share it without this implying infringements of intellectual property. Furthermore, in organizations with an active information technology team, open source software can be fully adjusted to the reality and desires of the institution or team that is in charge of managing the processes, which makes the use of the BPMS individualized and personalized, providing a more satisfying experience.

In view of this, five combinations of desired attributes were established for the evaluation of the type of license, expressed in the following conditions (listed here in descending order of preference): 1) totally free and free license; 2) completely free and proprietary license; 3) partially free and free license; 4) partially free and proprietary license; 5) Fully paid and proprietary license.

Regarding Bonita, it was found that it presents itself as a software with a partially free license and a free type license, reaching the third level of the attributes evaluated in this criterion. According to information obtained on the Bonita BPM website, the company offers two versions of the system license, one free (called Community Edition) and a paid business version, called Enterprise Edition. In addition, BPMS Bonita presents itself as an open source software, a feature that the company itself presents as a distinctive element in relation to other BPMS available on the market.

On the other hand, in relation to Bizagi, it was possible to observe that this is a software that also has a partially free license, but of the proprietary type, which places it in the fourth level of the combinations chosen for the criterion. As informed by the company, Bizagi also currently has two versions: the Personal version (individual and free) and the Enterprise version (paid and aimed at companies and other organizations).

Therefore, after completing the analysis of Bonita and Bizagi according to the type of license, the following judgment matrix was built, shown in table 6 below:

Table 5
Judgment matrix for the license type criterion

License type	Bizagi	Bonita
Bizagi	1	1/3
Bonita	3	1

Source: Elaborated by the authors (2022)

This matrix indicates that the prevalence of Bonita's performance compared to the performance obtained by Bizagi in the same criterion was only moderate.

b) Second criterion: simulation tool

Regarding this aspect, both the existence of the simulation tool in the two BPMS and the functionalities that the simulation, if available, offers to users, were analyzed. The simulation presents itself as a quantitative technique of process analysis that, according to Dumas et al. (2018, p. 279), is based on “using the process simulator to generate a large number of hypothetical instances of a process, executing these instances step by step and recording each step of this execution” (author translation). The existence of a simulation tool in BPMS is a desired feature as it expands the possibilities of process management by enabling successive tests in its instances and parameters, which makes the procedure more specific and efficient.

It should be noted, however, that it is possible that the BPMS does not offer the simulation tool in all its versions, restricting this functionality to only some specific or even paid versions. Considering, therefore, that the BPMS analyzed here have different versions (as analyzed in the previous criterion), this is a possibility that was considered worthy of mention.

Four qualifications were then elaborated that primarily assess whether or not the tool is made available by the BPMS, as well as whether this availability occurs in an unrestricted manner in all its versions. Again, the attributes will be listed in descending order of preference: 1) it offers simulation in all versions; 2) it offers the full simulation in the paid version and partially in the free version; 3) offers the simulation only in the paid version; 4) does not offer simulation in any version.

Regarding the analysis of BPMS in terms of this attribute, it was found that Bonita does not offer the simulation tool in any of its versions. According to the searches carried out as a result of this research, it was possible to verify that the last version of Bonita BPM with support for the simulation tool was version 7.3.3, released in 2016.

Continuing with the analysis of the BPMS, we began to investigate the presence of the simulation tool in the BPMS Bizagi. In fact, until the beginning of 2022, Bizagi Modeler offered the simulation tool integrated into its system, but through the Professional version, which was a paid individual version. However, this version was recently removed from the company's catalog, so that only the free individual version (Personal) and the paid version aimed at companies and organizations (called Enterprise) remain. Therefore, currently the Bizagi simulation tool is only present in the paid version and intended for companies, which makes it impossible for individuals to acquire this license individually.

However, it was possible to partially use the simulation tool, available in Bizagi Studio (version 12.0.1) - one of the components that, alongside Bizagi Modeler and Bizagi Automation, integrate the Bizagi platform. Therefore, although the simulation tool is not fully available in its free version, Bizagi still has an advantage over Bonita in this criterion, since it offers the tool partially for free. Furthermore, if an organization chooses to purchase the Bizagi business license, it will be able to fully use the simulation functionality. Thus, the judgment matrix for this criterion is presented as shown in the following table:

Table 6
Judgment matrix for the simulation tool criterion

Simulation tool	Bizagi	Bonita
Bizagi	1	5
Bonita	1/5	1

Source: Elaborated by the authors (2022)

It can be seen then that Bizagi, classified in the second level of the criterion, presented a strong performance superiority in relation to Bonita.

c) Third criterion: user support

For this criterion, we initially examined the existence of support, the category in which this support is provided, and the breadth/quality of user support, a factor that is quite relevant to the user experience. The quality and breadth of assistance offered by software developers brings security, reliability and ease of use of the systems. Furthermore, user satisfaction is related to the quality and extent of support received after obtaining the product or service (GOFFIN and NEW, 2001).

The following attributes were then established to evaluate the extent and type of support offered by Bizagi and Bonita according to the versions/licenses available: 1) personalized support in all versions; 2) personalized support only in the paid version and support via forum/user communities in the free version; 3) support via forum/user communities only

In this regard, it was verified that both Bizagi and Bonita offer personalized support to users only in their respective paid versions. Users of the free versions, in turn, have the option of support through the forums, which are collaborative. In other words, in these forums there is a sharing of information, questions, answers, ideas and problems that cover the community formed by BPMS users and some representatives of the development companies, who eventually answer some of the doubts presented there.

On the other hand, both software have consumer support packages in their paid versions, which offer a range of services capable of meeting the most diverse demands of their consumers in a fast and personalized way.

Due to the characteristics exposed here, it was found that both BPMS have an equivalent level of consumer support, which is more limited in relation to the free versions of the two software and presents a greater level of improvement, customization and promptness in their paid versions. For this reason, in the present criterion, there is no significant distinction between Bizagi and Bonita, which configures the following judgment matrix:

Table 7
Judgment matrix for the user support criterion

User support	Bizagi	Bonita
Bizagi	1	1
Bonita	1	1

Source: Elaborated by the authors (2022)

As both BPMS presented the same level of performance in the user support criterion, the judgment matrix presents itself with indifference between them in the criterion.

d) Fourth criterion: usability

The concept of usability is properly presented by the Brazilian Association of Technical Standards (ABNT) in NBR 9241-11, produced based on ISO 9241-11:1998. The definition is given in order to clarify that usability can be understood as the “Extent to which a product can be used by specific users to achieve specific goals with effectiveness, efficiency and satisfaction in a specific context of use” (ABNT, 2002, p. 3). Thus, the usability criterion is intrinsically subjective, taking into account the interaction between the user and the software, a factor that can be influenced by several aspects since not every user has the same level of familiarity with certain manifestations of information technology.

For the usability examination, indicators such as: ease of use, intuitiveness, layout, fluidity, availability of tools, among other aspects capable of demonstrating the idea of ease and fluency in the use of the product were used, contributing to the general experience of using the BPMS. As a result, the overall usability experience of Bizagi and Bonita was classified into expressions of approval or disapproval, listed below: 1) Excellent; 2) Very good; 3) Good; 4) Reasonable; 5) Bad.

Regarding usability, both BPMS presented an excellent performance, reaching the best classification for the criterion. In fact, both Bizagi and Bonita are software whose handling is intuitive, presenting uncomplicated layouts and allowing the user to easily find the necessary elements to model the processes, quickly understanding their meanings.

In addition, the presentation of the software is clear, with the tools and options arranged in a simplified way, so that specific technical knowledge is not required for the user to feel able to handle the system.

With that, the general conception about the usability of the two BPMS analyzed is in the sense that they reached the maximum level of performance, considering that none of them posed major obstacles for use, even in the first contact. The judgment matrix, presented in the table below, looks like this:

Table 8
Judgment matrix for the usability criterion

Usability	Bizagi	Bonita
Bizagi	1	1
Bonita	1	1

Source: Elaborated by the authors (2022)

Again, as it is a criterion in which both BPMS have demonstrated an equivalent performance, the judgment is in the sense that the importance of the two is identical, with no significant preference in favor of one or the other.

e) Fifth criterion: languages

The next criterion used as an object of analysis is related to the language of BPMS, an attribute that is capable of facilitating or hindering its use. In this sense, despite the wide diffusion of the English language - especially when it comes to information and communication technology instruments -, its mastery is still not something within the reach of all Brazilians - according to the British Council (2014), until the year 2013, only 5.1% of the Brazilian population over 16 years old declared that they could speak English.

Therefore, the offer of the system in Portuguese was considered positive, which suppresses any language barrier for users of organizations located in Brazil. Considering, however, the reasonable linguistic proximity between Portuguese and Spanish, it was considered a satisfactory alternative (although not entirely appropriate) to have the system available in Spanish, a language that provides a good level of textual understanding to Portuguese speakers (HENRIQUES, 2000).

Therefore, combinations were created that express the mentioned properties, that is, the language and the form of its presentation, demonstrating if it is limited to the software or if it also extends to the website, in whole or in part, involving the other related environments. to the program, as the user may occasionally access them in search of information. The attribute associations generated the following classifications: 1) BPMS and website entirely available in Portuguese; 2) BPMS fully available in Portuguese, site partially available in Portuguese; 3) BPMS entirely available in Portuguese, site entirely available in Spanish; 4) BPMS entirely available in Spanish, site entirely available in Spanish; 5) BPMS and website fully available in English or another language (except Portuguese and Spanish).

Regarding BPMS Bizagi, it was found that both its environment and website are available in Portuguese, which is beneficial to Brazilian users, who do not need to resort to external devices to navigate the system or access its resources. However, Bizagi did not reach the highest rating because some options on the site are only offered in Spanish and English, as is the case with the user community - which includes the forum with doubts, ideas, questions, etc. - and documentation. With that, Bizagi reached the second classification of the scale.

On the other hand, Bonita has an even greater language restriction than that found in Bizagi. Although Bonita Studio software is available in Portuguese, its website and all official information regarding BPMS are offered in English, Spanish and French. Thus, any resource external to the software platform is, at best (for non-English speaking Brazilians), available in Spanish.

With this, we have that Bizagi has an advantage in this criterion by offering fewer linguistic obstacles than Bonita, considerably facilitating the use and providing an overall more accessible experience. The judgment matrix resulting from the analyzes is shown in table 9:

Table 9
Judgment matrix for the language criterion

Language	Bizagi	Bonita
Bizagi	1	3
Bonita	1/3	1

Source: Elaborated by the authors (2022)

It appears that for this criterion the degree of superiority of Bizagi's performance in contrast to Bonita's performance was considered moderate.

f) Sixth criterion: training

The evaluation of the training criterion had as a premise to verify if the companies that developed the BPMS offer courses or other forms of training to the user so that they can handle the tools, understanding their elements and their functionalities. The existence of preparation and learning platforms that can train users before the effective use of the tool is a factor that deserves to be considered, as it minimizes possible mistakes in the use of the systems. This criterion is intended to assess whether the solution provides tools to enable the user to use, navigate and take advantage of the software, making its use more efficient.

Taking these aspects into account, five combinations of traits considered relevant for the analysis of this item were listed: 1) Free courses or training available at all times; 2) Free courses or training available occasionally or on demand; 3) Paid courses or training available at all times; 4) Paid courses or training available occasionally or on demand; 5) Lack of courses or training.

The analysis of this criterion brought another point of distinction between the practices adopted by the companies that developed the two BPMS under study. Regarding Bizagi, we found that it is possible to find a wide catalog of courses on various topics, several of which are aimed at the practical use of the modules that make up the Bizagi platform (Modeler, Studio and Automation), which are immediately available.

Bonita, in turn, does not have a permanent course catalog, and it is possible to request training aimed at the Bonita Studio platform only for those users who have contracted the Enterprise version, that is, the paid business version.

Due to the characteristics exposed above, the Bizagi obtained the maximum classification in the category, while the Bonita received the penultimate classification. With this, the judgment matrix was prepared and demonstrated through table 10:

Tabela 10
Judgment matrix for the training criterion

Training	Bizagi	Bonita
Bizagi	1	7
Bonita	1/7	1

Source: Elaborated by the authors (2022)

The matrix presented this arrangement due to the considerable difference in performance of the two BPMS, so that Bizagi had its performance evaluated with a very strong level of superiority in relation to what was demonstrated by Bonita in the same criterion.

g) Seventh criterion: documentation

Finally, the documentary collection offered by BPMS was analyzed, that is, the set of information that is made available by companies about the software, covering an overview of the systems, user manuals, frequently asked questions and answers guides, forms, information about security etc. The variety, breadth and ease of access related to this documentation increase the autonomy of the user, who can resort to the information they need without the obligation to call support or wait for mediate solutions that can delay process management activities.

This instrument becomes even more useful in the case of software whose consumer support is limited or paid - as is the case with the BPMS analyzed here -, since the documentation works as a kind of encyclopedia of the system, containing various information about the software.

In order to evaluate the documentation, several aspects were verified that reveal its general quality, such as scope, level of detail, clarity and organization (that is, if the documentation is dispersed or, on the other hand, if it is concentrated in a specific location, with the proper ordering). Thus, the quality of the documentation was classified into five categories listed below: 1) Excellent; 2) Very good; 3) Good; 4) Reasonable; 5) Bad.

In terms of documentation, both BPMS showed a high performance, reaching the “excellent” rating. In this sense, it was found that Bizagi and Bonita provide detailed and complete documentation, containing a wide variety of information regarding the systems and how to use them, from installation to the first steps to use the systems. For this reason, the two BPMS received five points in this criterion, which led to the elaboration of the judgment matrix shown in the following table:

Table 11
Judgment matrix for the documentation criterion

Documentation	Bizagi	Bonita
Bizagi	1	1
Bonita	1	1

Source: Elaborated by the authors (2022)

The arrangement of this matrix repeats what was presented in relation to the criteria “user support” and “usability”, in which it was not possible to verify differences in the performance of the alternatives, thus showing the same degree of importance between them.

Once the performance analysis step was completed, it was possible to carry out the last phase of the AHP execution, aggregating the partial results in a global analysis, considering the weights of each criterion. Therefore, the partial performance indicated in the previous step needs to be weighted and, in the end, condensed into a final result capable of indicating which of the alternatives has the highest level of global priority, considering the joint performance in all criteria (and with due regard weight assigned to these criteria).

In order to carry out this step, it was necessary to submit the judgment matrices to normalization and consistency checking procedures. After executing the procedures and calculations with the aid of Microsoft Excel, it was possible to obtain the following priority matrix for the alternatives according to the criteria (still without the assignment of weights), as shown in table 12:

Table 12
BPMS local priority matrix for the analyzed criteria

BPMS	License Type	Simulation tool	User support	Usability	Language		Training	Documentation
Bizagi	0,250	0,833	0,500	0,500	0,750	0,875		0,500
Bonita	0,750	0,167	0,500	0,500	0,250	0,125		0,500

Source: Elaborated by the authors (2022)

With the matrix of local priorities built, there remains the need to weigh the results and, finally, aggregate them. The weighting of the values indicated in the matrix of local priorities is premised on the idea that the criteria do not have the same weight for the decision-making problem, so that the performances cannot be analyzed without the incidence of priority vectors. For example, the “grade” obtained by Alternative A when its performance in Criterion 1 is verified must be multiplied by the weight that this criterion has for the decision problem. Thus, the weighting procedure allows the result obtained with the use of the method to truly reflect the preferences of the decision maker, whose predilection for one criterion over another will exert direct effects on the final added value.

Consequently, after the incidence of weights on the results, the matrix of local priorities was presented as shown in table 13 below:

Table 13
BPMS local priority matrix after the incidence of weights

BPMS	License Type	Simulation tool	User support	Usability	Language	Training	Documentation
Bizagi	0,012	0,152	0,038	0,170	0,021	0,200	0,049
Bonita	0,035	0,031	0,038	0,170	0,007	0,029	0,049

Source: Elaborated by the authors (2022)

Finally, in order to obtain the final priorities of the two BPMS in the face of the decision-making problem, the sum of the values found for each of the criteria was performed, thus reaching the following results, indicated in table 14:

Table 14
BPMS final priorities.

Alternative/BPMS	Final priorities
Bizagi	0,642 (64,2%)
Bonita	0,357 (35,7%)

Source: Elaborated by the authors (2022)

With this, the execution steps of the AHP method were completed, resulting in a scenario in which the BPMS Bizagi presented itself with a priority degree of 64.2%, while Bonita had a preference performance of 35.7%, indicating that Bizagi was the software that brought the best performance in the presented questions.

5 FINAL CONSIDERATIONS

This article aimed to conduct a process for analysis and decision making in order to compare and enable the choice of one of two tools for mapping information flows, modeling and simulation of processes in BPM: the BPMS Bizagi and Bonita. For this purpose, specific techniques of the multicriteria decision analysis methods (MMAD) were used, more specifically the Analytic Hierarchy Process (AHP) method.

In view of the results presented, it was possible to reach the solution of the decision problem between the BPMS Bizagi and Bonita, having as an outcome the verification that Bizagi had a higher degree of priority than Bonita. However, it is necessary to make some considerations about these results.

It is known that the decision-making process corresponds to an equation that involves several elements, whose interplay can vary drastically. Thus, no two decision-making processes are the same, and even if the elements are identical, the final result can change according to the place, time or context in which the decision needs to be made.

In short, the decision-making process is naturally subjective. Although several decision-making methods with quantitative biases have been created, capable of minimizing the inaccuracies resulting from this subjectivity, qualitative data and their own relativity and mutability cannot be neglected.

That said, the results obtained with the use of the AHP in this research are not intended to be definitive, since they are contingent on the basis of the experiments carried out. It is also necessary to emphasize that the result presented is consistent with the priority vectors that were established. Therefore, BPMS Bizagi performed significantly better than BPMS Bonita in two of the three most important criteria, namely, the training criterion and the simulation tool criterion.

It should also be noted that although the criterion with the highest relative priority was usability, both BPMS presented the same level of performance in this category, which made the criterion, although relevant, not have a great impact on the calculation of final priorities.

Furthermore, it is worth noting the fact that BPMS Bonita showed a better performance than BPMS Bizagi in terms of the type of license, since the former presents itself as an open source software. However, considering that this criterion was not given a very expressive weight, the preponderant performance of Bonita was not enough to raise its final classification in relation to the established priorities.

All this contextualization serves to reinforce that the final result of this article does not intend to indicate an absolute superiority of Bizagi in relation to Bonita. The results could have been different if the priority judgment in relation to the criteria had been different or if, in the same way, other criteria had been taken into account. This addendum, however, does not invalidate the stated objective, since the premise was to analyze the BPMS under equal conditions, comparing them and verifying their performance in light of the criteria considered to be the most relevant.

Thus, for the criteria that were used as a parameter and given the weights that were distributed to them, Bizagi proved to be the preferable BPMS for conducting all the steps for mapping information flows, modeling and managing the selected process. The result presented here serves as a form of recommendation for other organizations that seek to manage and improve similar business processes. Furthermore, it allows new studies to be developed and applied in different processes, or in other sets of BPMS tools. In this specific case, Bizagi presented a higher score, without disregarding the positive points of Bonita, which may be seen as a priority depending on the scenario or the individuals interested in the process.

• Availability of data and material:

  The data can be accessed as soon as the process of depositing the Dissertation that originated this article is in the UFPB repository.
• Funding: Not applicable.

REFERÊNCIAS

• ABPMP (Brasil). BPM CBOK. Guia para o gerenciamento de processos de negócio: corpo comum de conhecimento. ABPMP BPM CBOK, v. 3.0, ABPMP. Brasília, 2013. Disponível em: https://bit.ly/3QSDua2 Acesso em: 25 maio 2022.
  » https://bit.ly/3QSDua2
• ARAÚJO, W. J et al Validação do mapeamento de fluxos de informação em processos organizacionais: uma abordagem com foco arquivístico. In: ENCONTRO NACIONAL DE PESQUISA E PÓS-GRADUAÇÃO EM CIÊNCIA DA INFORMAÇÃO, 19, 2018, Marília. Disponível em: https://bit.ly/3N9TtzY Acesso em: 20 jul. 2022.
  » https://bit.ly/3N9TtzY
• ASSOCIAÇÃO BRASILEIRA DE NORMAS TÉCNICAS. ABNT NBR 9241-11: Requisitos ergonômicos para trabalho de escritórios com computadores, parte 11 - orientações sobre usabilidade. Rio de Janeiro, 2002. Disponível em: https://bit.ly/3FnoPRQ Acesso em: 31 maio 2022.
  » https://bit.ly/3FnoPRQ
• BIZAGI. About Bizagi. Washington, DC, 2022. Disponível em: https://bit.ly/3fbgBS9 Acesso em: 17 out. 2022
  » https://bit.ly/3fbgBS9
• BONINI, A. C. et al Um estudo teórico sobre a história da pesquisa operacional. Cadernos de Artigos Científicos e Resumos Expandidos, Curitiba, v. 1, n. 4, p. 1666-1670, jun. 2016. Disponível em: https://bit.ly/3FmbMjG Acesso em: 29 jan. 2022.
  » https://bit.ly/3FmbMjG
• BONITASOFT. What is Bonita? Boulogne-Billancourt, 2021. Disponível em: https://bit.ly/3W8eI9C Acesso em 17 out. 2022.
  » https://bit.ly/3W8eI9C
• BRITISH COUNCIL. Learning English in Brazil: understanding the aims and expectations of the Brazilian emerging middle classes. São Paulo, 2014. Disponível em: https://bit.ly/3TZV9OF Acesso em: 9 jun. 2022.
  » https://bit.ly/3TZV9OF
• CHOO, C. W. A organização do conhecimento: como as organizações usam a informação para criar significado, construir conhecimento e tomar decisões. São Paulo: SENAC, 2003.
• COSTA, H. G. Introdução ao método de análise hierárquica: análise multicritério no auxílio à decisão. Niterói: UFF, 2002. Disponível em: https://bit.ly/3Wbxz3E Acesso em: 18 jun. 2022.
  » https://bit.ly/3Wbxz3E
• DUMAS, M. et al. Fundamentals of business process management. 2nd ed. Berlin: Springer, 2018.
• GOFFIN, K.; NEW, C. Customer support and new product development - an exploratory study. International Journal of Operations & Production Management, Bingley, v. 21, n. 3, p. 275-301. Disponível em: https://doi.org/10.1108/01443570110364605 Acesso em: 31 maio 2022.
  » https://doi.org/10.1108/01443570110364605
• HENRIQUES, E. R. Intercompreensão de texto escrito por falantes nativos de português e de espanhol. Revista de Documentação de Estudos em Linguística Teórica e Aplicada, São Paulo, v. 16, n. 2, p. 263-295, 2000. Disponível em: https://doi.org/10.1590/S0102-44502000000200003 Acesso em: 09 jun. 2022.
  » https://doi.org/10.1590/S0102-44502000000200003
• LACHTERMACHER, G. Pesquisa operacional na tomada de decisões. 5. ed. Rio de Janeiro: LTC, 2016.
• LEHNHART, E. R. Tomada de decisão em contextos específicos: uma análise das relações entre os métodos multicritérios de apoio à decisão e as estratégias de decisão humana. 2016. 267 f. Tese (Doutorado em Administração) - Universidade Federal de Santa Maria, Santa Maria, 2016. Disponível em: https://bit.ly/3zkSYgF Acesso em: 04 jan. 2022.
  » https://bit.ly/3zkSYgF
• MATEO, J. R. S. C. Multi-Criteria Analysis. In: MATEO, J. R. S. C. Multi-criteria analysis in the renewable energy industry. London: Springer, 2012, cap. 2, p. 7-10. Disponível em: https://doi.org/10.1007/978-1-4471-2346-0_2 Acesso em: 30 jan. 2022.
  » https://doi.org/10.1007/978-1-4471-2346-0_2
• MORAES, E. A.; SANTALIESTRA, R. Modelo de decisão com múltiplos critérios para escolha de software de código aberto e software de código fechado. Revista Organizações em Contexto, São Bernardo do Campo, v. 4, n. 7, p. 59-83, jun. 2008. Disponível em: https://bit.ly/3SFy2Im Acesso em: 18 jun. 2022.
  » https://bit.ly/3SFy2Im
• RIOS, I. R. Análise de fluxos informacionais do processo de aquisição por pregão eletrônico da Pró-Reitoria Administrativa da Universidade Federal da Paraíba 2019. 211 f. Dissertação (Mestrado em Ciência da Informação) - Universidade Federal da Paraíba, João Pessoa, 2019. Disponível em: https://bit.ly/3N9yGg3 Acesso em: 16 ago. 2022.
  » https://bit.ly/3N9yGg3
• ROY, B. Multiple criteria methodology for decision aiding Dordrecht: Springer Science, 1996. E-book. Disponível em: https://doi.org/10.1007/978-1-4757-2500-1 Acesso em: 27 jan. 2022.
  » https://doi.org/10.1007/978-1-4757-2500-1
• SAATY, T. L. Decision making - the analytic hierarchy and network processes (AHP/ANP). Journal of Systems Science and Systems Engineering, Berlin, v. 13, n. 1, p. 1-35, mar. 2004. Disponível em: https://doi.org/10.1007/s11518-006-0151-5 Acesso em: 14 jun. 2022.
  » https://doi.org/10.1007/s11518-006-0151-5
• STALLMAN, R. Why “open source” misses the point of free software. Communications of ACM, New York, v. 52, n. 6, p. 31-33, jun. 2009. Disponível em: https://doi.org/10.1145/1516046.1516058 Acesso em: 28 maio 2022
  » https://doi.org/10.1145/1516046.1516058
• VARGAS, L. G. An overview of the analytic hierarchy process and its applications. European Journal of Operational Research, North-Holland, v. 48, n. 1, p. 2-8, 1990. Disponível em: https://doi.org/10.1016/0377-2217(90)90056-H Acesso em: 27 jun. 2022.
  » https://doi.org/10.1016/0377-2217(90)90056-H
• WESKE, M.; VAN DER AALST, W. M. P.; VERBEEK, H. M. W. Advances in business process management. Data & Knowledge Engineering, Amsterdam, v. 50, n. 1, p. 1-8, 2004. Disponível em: https://doi.org/10.1016/j.datak.2004.01.001 Acesso em: 03 mar. 2022.
  » https://doi.org/10.1016/j.datak.2004.01.001
• YU, A. S. O. Tomada de decisão nas organizações. São Paulo: Editora Saraiva, 2011.

Data availability

The data can be accessed as soon as the process of depositing the Dissertation that originated this article is in the UFPB repository.

Publication Dates

• Publication in this collection
  23 Jan 2023
• Date of issue
  2022

History

• Received
  23 Aug 2022
• Accepted
  13 Oct 2022
• Published
  06 Nov 2022