Development of a measurement instrument to evaluate integrated management systems and differences in perception: an approach to item response theory and the quality management process

Abstract

Paper aims

The first aim is methodological, by developing a conceptual model to describe the internal relationship environment (IRE), the critical factors that impact this environment, the characterization of the parties involved, and their relationships. The second is practical and instrumentalizes the model to measure the effect of differences perceived by internal customers.

Originality

Distinct works focus on the formulation of management systems, successful implementation, or external and market environmental factors, although there is a lack of studies that relate organizational performance to differences in perceived quality between the parties.

Research method

The methodology followed a flow of collection/analysis, of the informational data of the company, sketch of the model and flow of information, exploratory focus group, thematic analysis of content, and confirmatory focus group. Then, the procedure of operationalization of the model.

Main findings

The conceptual model and its instrumentalization describe the apparent relationships between the support team and the operations teams, the underlying relationships of the ERI with the company's management model, and organizational performance.

Implications for theory and practice

In practice, the proposed measurement instrument allows evaluation of the effects of differences in the perceived quality of internal customers.

Keywords
Assessment requirements; Business management models; Dimensions of quality; Item response theory; Organizational excellence

1. Introduction

In the mining industry context, special consideration is given to the short-term production schedule to ensure compliance with the production targets and restrictions imposed by the long-term plan (Quigley & Dimitrakopoulos, 2020; Sousa et al., 2022). In addition, given the specificities of the exploration process, it is necessary to have efficient production planning and control (PPC) combined with the moderating factors of equipment maintenance, the latter exercised by the maintenance programming and control team (MPC) to ensure availability, reliability, and maintainability of the equipment. Thus, efficient integration between PPC and MPC tends to ensure that production is carried out by product quality specifications and annual production targets and, consequently, results in excellent operational performance (Fernandes et al., 2021; Maheswari et al., 2020; Pascual et al., 2016; Qiao et al., 2017; Rocha et al., 2021).

Several scientific works were proven as presented these aspects, subject to specialists of different quality and reliability (Bouslah et al., 2016; Hadidi et al., 2012). Thus, an urgent challenge for scholars and professionals in Business Process Management (BPM) is how to adapt production operations to respond to the transition to an industrial environment with widespread digital technologies. (Bokrantz et al., 2020).

In the context of Industry 4.0, technological development aims to enable the acquisition of measurable process parameters and acquisition of information directly from the equipment, thus being the process capable of integrating multiple devices into an integrated system, collecting data from the entire infrastructure and environment, as well as improved measurement and management of the mine's spatial and intertemporal compliance with the plan in open-pit mines in a holistic and integrated manner (Gackowiec et al., 2020; Otto & Musingwini, 2020).

Given the above and considering the concepts of Industry 4.0, practices of Total Quality Management (TQM), and organizational performance, the latter, operationalized in a multidimensional way, three key issues in the mining context are raised: [1] that the exploration project considers the life of the mine, combined with the selling price minus the cost of exploration, thus establishing a profit perspective that determines the viability or not of the project (Drummond et al., 2017; Wårell, 2018); [2] the choice of the manufacturing process that mainly takes into account the economic criterion given by the lower unit cost when considering all operating constraints (Barratt & Ellem, 2019; Sane, 2018; Shen et al., 2018) and; [3] an integrated management system (IMS) capable of providing an unfolding of strategies, which results in measurable indicators and can be applied in specific sectors of operation. (Chatterjee et al., 2016).

The company under study is a Brazilian multinational and is among the largest global mining companies, whose management system is well formulated and integrated, and despite being in constant evolution, it finds difficulties in the process of deep and comprehensive implementation in all teams of operation and represents a critical factor for the company under study. In this context, when considering that this process is a top-down approach, the research gap is the realization of the evaluation by the bottom-up approach, that is, it seeks to evaluate, from the perspective of lower hierarchical levels, the ability to disseminate strategies through the IMS adopted by the company.

Specifically, when treating operational excellence as the ability of the operations team to ensure that annual production targets are met, the objectives are formalized in the following research questions (QR):

RQ₁: How does the IMS dissemination strategy affect operational performance?

RQ₂: How the maturity of each team affects operational performance?

RQ₃: What are the critical factors that impact the IMS implementation process?

RQ₄: How team perception affects the IMS implementation process?

2. Theoretical foundation

2.1. Integrated management system

Corresponding to the strategic implementation and execution process and focusing on organizational excellence, an integrated management system (IMS) (Srivastava & Sushil, 2015), consists of deploying strategies that result in measurable indicators that can be applied in specific sectors of operation. Furthermore, according to (Asif et al., 2011), IMS assessment involves setting goals, selecting appropriate measures, collecting data related to goal achievement, and using the data for organizational improvements. Consequently, it develops potential benefits, such as improving communication between the levels and functions of the organization, improving the coordination of the implementation and execution processes of the IMS, and improving the direction of people in promoting the desired results.

In the context of mining, according to (Gackowiec et al., 2020, p. 2):

Increasing the efficiency of the conducted processes is invariably the major objective of mining companies. Efficient processes determine the efficiency of the entire organization and are dependent on the established management system in the company. The crucial issue at this point is to know how to make use of the collected data to improve the performance of the work.

In a more general scope of Quality Management (QM), according to (International Organization for Standardization, 2020, pp. 17-19), the implementation of a quality management system (QMS) is “[...] the process of establishing, documenting, implementing, maintaining and continually improving a QMS [...]” and, additionally, the concept of the management system is “[...] set of interrelated or interacting elements of an organization to establish policies and objectives, and processes to achieve those objectives [...]”. So, four points stand out:

1. Elements are functions of organizational performance dimensions that, according to (Xu et al., 2020), are operationalized as a multidimensional construct, including financial performance, operational performance, customer service, and product quality;

2. Interrelated or interactive, (Hadidi et al., 2012) make a distinction between these concepts, namely: interrelated models are those in which only as decision variables of a relation are functions in functions, taking into account other functions like, while integrated models are those in which two functions are modeled and optimized simultaneously;

3. Establish policies, objectives, and processes, that is, each dimension of analysis constitutes the formulation of specific goals, the selection of appropriate measures, the collection of data related to the accomplishment of the goals, and the use of the data for institutional improvements (Banta et al., 2009);

4. Objectives mining strategies, strategically, refer to the exploration project, the choice of the manufacturing process, and the integrated management system (IMS) adopted. However, according to (Fernandes et al., 2021), organizational performance interests can be classified in four ways, not restricted, in terms of quality objectives, namely: [1] Reliability of information, [2] cost reduction production, [3] iron ore content and, finally, [4] flow capacity.

Likewise, this understanding coincides with the concept of Quality 4.0 adopted by (Sisodia & Villegas Forero, 2019), in particular:

Quality 4.0 refers to the digitalization of TQM and its impact on quality technology, process, and people. It builds upon traditional, quality tools and considers connectedness, intelligence, and automation to improve performance and make timely data-driven decisions in an end-to-end scenario, involving all the stakeholders and providing visibility and transparency.

Otherwise, by associating the concept of Sustainable Development (SD), a considerable volume of Sustainability Indicators (IS) was proposed, making the process of adapting production operations a challenge for scholars and business managers, to respond to the transition to an industrial environment conditioned to sustainability concerns, in which there is a growing effort to consider optimal solutions that integrate the economic, social and environmental dimensions (Triple Bottom Line - TBL) subject to the loss of financial and natural resources, bad reputation and the potential environmental risks and degradations (Fernandes et al., 2021; King, 2016; Mohsin et al., 2021; Nairn et al., 2020; Nicholls, 2020; Silva et al., 2021).

In addition, (Asif et al., 2011, p. 361):

The integration of sustainability into business processes requires continuous interaction with stakeholders, and innovative ways of designing, reviewing, and updating business processes. It is very dynamic nature requires the development of internal competencies and the institutional knowledge to deal with emerging issues and their impacts on the organization.

Correspondingly, this problem becomes complex when considering the multiple perspectives of stakeholders, environmental protection regulations, society's growing awareness of the importance of sustainable development, and the prevalence of a volatile and competitive market environment. (Belotti Pedroso et al., 2021; Pan et al., 2021).

2.2. IMS Quality Assessment Process

Practically, when incorporating strategic objectives, a substantial amount of data (internal/external) will be available to companies to support the management of the business, being, in this way, essential to the efficient formulation and implementation of the sustainable business model (SBMs), these that necessarily better translate strategic objectives into measurable elements (Battilana et al., 2020; Bernardo et al., 2022). Therefore, given that the concept of “Quality is the “[...] degree to which a set of inherent characteristics of an object fulfils requirements [...]” (International Organization for Standardization, 2020, p. 21), and a specified requirement must be declared, implicit and mandatory, these measurable elements are the IMS assessment requirements.

In this sense, the word “Assessment” is similar to the expression “Internal Quality Assurance (IQA)”, in that the objective of an IQA system, similar to the main certification systems, is to provide evidence of accountability to the interested parties while ensuring reliable processing of the information generated, as well as allowing the company to function as efficiently as possible.

However, as (King, 2016), show three drivers of effective measurement systems: [1] improved organizational capacity that, associated with the assessment of operational performance through KPIs, is essential to understand the improvements or failures in the evolution of the IMS; [2] consistent high-level support that, through maturity assessment, is the basis for defining work programs and achieving organizational excellence.

3. Measuring organizational performance

3.1. Key Performance Indicators (KPIs)

Typically, operational performance is measured by Key Performance Indicators (KPIs), which allow process evaluations by comparing practical performance (actual) versus the programmed target (estimated) (Parmenter, 2015; Peral et al., 2017). To this end, works such as (Gackowiec et al., 2020) reviewed the KPIs used for monitoring processes in the mining industry, classifying them into different dimensions (technical, temporal, spatial, safety and maintenance) for different groups of people (superintendent, shift supervisor, operator, and maintenance manager). Otherwise, considering the series of standards ( International Organization for Standardization, 2014a , b , 2017 , 2018) that specify the main KPIs used in manufacturing operations management, (Kang et al., 2016) presents a framework of relationships between KPIs and their supporting elements, and (Bhadani et al., 2020) analyzes KPIs for aggregate production using dynamic simulation.

In addition to the perspective of technological and competitive evolution, strategic management elements linked to the objectives of companies should guide how they distribute their resources in the production process (Mohammadi et al., 2017; Nairn et al., 2020; Nunes et al., 2019; Pereira & Nunes, 2018; Santos et al., 2020). In other words, strategic management elements establish structured management routines, methodologies, and tools to sustain and improve results, and are essential to solve problems and ensure the principle of continuous improvement, that is, failure to meet the schedule by the KPIs constitutes the identification and prioritization of improvement opportunities in which their support elements point to improvement opportunities.

Otherwise, Business Process Management (BPM) constitutes a reference for an efficient operation process (Lee & Dale, 1998). Thus, the strategic objectives are disaggregated down into criteria, which are the expression of the states that describe the states or requirements of the mining macro process. Verification of compliance with each service determination is established using a set of specific indicators or practices, which can be verified or qualitative. Thus, by a hierarchy of indicators, quality reference of the evaluation process, and operational evaluation of implantation and execution of a model of implantation of the implantation quality and control of processes for implantation of the IMS.

3.2. Maturity assessment process

The production process in mining refers to an aggregate production plant, which consists of distinct process operations, each operated with multiple objectives for plant operators and managers. Thus, the complexity of implementing the IMS increases, and demand for an integration of management tools that better translates strategic objectives into measurable elements at the operation level (Bernardo et al., 2022; Bhadani et al., 2020).

The entire organization’s efficiency is the set of efficiencies in each process, considering all organizational dimensions together. Related to operational processes, operational excellence is directly measured by the achievement of annual production goals and determines the improved organizational capacity that, associated with the evaluation of operational performance through KPIs, is essential to understand the improvements or failures in the evolution of the IMS (Peral et al., 2017).

Combining internal and external issues for each process operation, consistent high-level support that, through maturity assessment, is the basis for defining work programs and achieving organizational excellence and is a second driver for measurement systems effective. To this end, the measurement of the IMS, adopted by the company under study, is carried out by the Maturity Assessment which, inspired by Dupont's Bradley Curve, will determine the maturity level of the operation process and is categorized into 5 stages (0 - non-existence or non-implemented practice, 1 – weak or beginning to be implemented, 2 – being implemented, but not yet comprehensive, 3 – implemented and achieving expected results, 4 – excellence or internalized and continuously improved).

Otherwise, dependence on the IMS established in the company, introduces the need to evaluate the process of implementation and execution of the IMS itself, which, in short, must consider the following steps: [1] deployment and communication of the strategy that, with a view to an organizational alignment in which everyone understands their role, refers to the process of converting strategic objectives into measurable elements at all organizational levels (Srivastava & Sushil, 2017); [2] Routine Management and aims to ensure that all operational areas continually analyze indicators, expose problems, align priorities, and take the necessary actions to achieve operational excellence.

3.3. Quality meta-assessment

To triangulate Measuring Organizational Performance in search of organizational excellence, the quality meta-assessment refers to the set of assessments to guide institutions towards best quality improvement practices (Ory, 1992). Likewise, it grants the venture the so-called reputational capital, which is primarily responsible for promoting the company's image to consumers, directly reflecting the organizational performance of the corporation, due to the sustainable model that the brand represents and takes to the market (Basso et al., 2020; Bulkan, 2020; Miles & Covin, 2000).

In general, measurements can be direct, which consists of a quantitative assessment, and indirect measures, which consists of quantitative and qualitative assessments. In addition, a second indirect measurement approach is a qualitative assessment and typically requires stakeholders to reflect on their performance.

In this perspective, as a way of specifying and contextualizing the concepts addressed, this study formalizes three processes of quality assessment (QA), namely:

QA₁: The assessment by KPIs is a direct and quantitative measure and will reflect the effectiveness of the operation. This understanding is associated with one of the 5 approaches to the definition of quality (Green, 1994), and quality is understood as effectiveness in achieving institutional;

QA₂: The maturity assessment, will determine the maturity level of the operation process and represents the evolution of the implementation and execution process of the IMS itself. Therefore, it is an indirect measure, but quantitative and qualitative, and will reflect the efficiency of the operation. This approach is associated with the concept of quality as meeting the stated needs of stakeholders and highlighting the importance of knowing who the stakeholders are, their needs, and how to satisfy them;

QA₃: The assessment of perceived quality, which is an indirect and qualitative measure, considers quality in its traditional sense and is aligned with a vision of quality par excellence (Elassy, 2015). Furthermore, internally, according to (International Organization for Standardization, 2020, p. 3), people are: [1] “The performance of the organization depends on how people behave within the system in which they work” e, [2] “Within an organization, people become engaged and aligned through a common understanding of the organization's quality policy and desired outcomes”.

3.4. Perceived quality assessment process

For this research, it can be summarized that most of the literature on organizational performance measurement, in the context of the mining industry, to date has focused on the figurative anatomy/structure of the IMS (Bernardo et al., 2022). Furthermore, according to (King, 2016), there is still a third driver for effective measurement systems: the recognition and awareness of problems and the point of greatest impulse as people become enlightened (Hezri & Dovers, 2006; Singh et al., 2009).

It is precisely the reference to the recognition and awareness of problems that constitute the basis for the development of an evaluation based on the perception of the employees' quality. Introducing quality differences, so, quality differences that can be realized for compliance, push targets that people's ability to meet can be executed, push targets on operations compliance level, push targets on compliance level task fulfillment. Perception regarding the inclusion of activities to direct operational activities and, in the case of management support equipment, through dissemination, instruction, training of management personnel among managers and contrast through the communication of the relationship involved.

Otherwise, given that perceived quality has an intangible characteristic, which according to (Juran, 1992), is the result of the interpretation of one or more characteristics/attributes that make up the thing of interest, it becomes quite complex to establish a consensual definition for what is intangible and that cannot be measured directly and objectively, mainly by the divergence in determining what integrates or determines the intangibles (Houaiss, 2001; Soligo, 2012). Thus, the difficulty in measuring intangibles largely lies in the definition of what to measure and how to face and encourage new ways of unraveling organizational problems.

4. Materials and methods

4.1. Application context

The motivation for this study arises from the proximity of the authors of this research to a large mining company. Such proximity allowed us to hear from employees, from different sectors and hierarchies, their needs, difficulties, and perceptions regarding the internal relationship environment, as proposed.

The work proposal was presented to the company and obtained the release for the study. Then, data and information were collected from the business management model, organizational structure, relationship dynamics and implementation, the process of information and demands generation, the process of information unfolding, identification of the parties, other documentary information, and the contact of a member of the support team to answer doubts and monitor the research. Based on these data, and together with the literature review, the conceptual models were outlined, and some factors were suggested as a starting point.

4.2. Data collection and analysis procedure

The data collection and analysis procedure followed a collection/analysis flow, thus, with the company's informative data and initial draft, two focus groups were conducted, in a controlled environment, with the participation of 1-2 members support team, and 3-5 operation team members.

Exploratorily, the first focus group was conducted in two phases: In the first, each participant reported their understanding of the process as a whole, as well as the responsibilities of their team, and in the second phase, based on the S-O-R theory, occurred the group dynamics where the participants were encouraged to react to factors and aspects based only on the experiences and suggestions of the authors of this article. To avoid conflict of interest or fear of participants in reporting their views, the mediators used the approach of the theory of approximation/avoidance, guiding the participants to a more generalist report and general understanding, while we sought to avoid reports or opinions personal. This second moment aims to identify the requirements of the parties involved, that is, it is related to the collection of the characterization of multiple perspectives, factors, and latent traits.

The information collected was analyzed by content theme which, according to (Braun & Clarke, 2006), offers an accessible and theoretically flexible approach to analyzing qualitative data. From this perspective, the collected information was transcribed, extensively and widely discussed, classified, and grouped. As a result of this step, the client's requirements were obtained, and a second version of the factors, conceptual model, diagram, and characterization of the parties involved was adjusted.

To describe the dynamic of the second focus group, it is necessary to understand the three types of content validation, which according to (Coluci et al., 2015; Pasquali, 2017a, 2020) achieved: [1] through the literature review, a theoretical basis and similarity to items validated in other measurement instruments; [2] validation of content confirmation with research participants; [3] validation by experts in the field, so that the minimum criteria necessary to validate the conceptual structure and set of items of each factor of the measurement instrument are sized.

Thus, in a confirmatory way, the second focus group was also performed in two phases: In the first, the conceptual models were verified, considering clarity, relevance or representativeness, and scope; then, each of the factors and the requirements raised in the first focus group was evaluated separately.

The resulting conceptual products are the IRE conceptual model, including impact factors, the information, and demands generation process diagram, and the characterization of the parties involved, thus achieving the methodological objective.

4.3. Internal relationship environment

Distinct theoretical contributions have allowed a better understanding of the latent trait and, consequently, have expanded the possibilities of application of this concept in different areas of study being, such as means of constructing measuring instruments in the form of scales and using a simple noun to describe the latent trait.

Concomitantly with these concepts, the IRT provides a methodological structure to measure constructs and latent traits, which have a characteristic of intangibility and, therefore, can be measured through a set of items, elaborated from a matrix of concepts or theoretical construct (Andrade et al., 2000; Pasquali, 2020).

In this study, we introduced the concept of internal relationship environment (IRE) as the place where a set of information on the relationships between supplier and internal customer of the organization is generated, that is, the union of these individuals within the corporate environment, to provide support and operation support. In addition, it aims to support the effective implementation of the company's management model, strengthening its values to leverage results toward continuous operational excellence. The development of this relationship is given through the adoption of a set of individual practices and the exchange of information about the products needed for support.

In the literature, the concept of IRE proposed is largely derived from the set of standards for quality management systems (International Organization for Standardization, 2020, 2015, 2021) and (PMBOK Guide, 2021). To a lesser extent, when relating to the quality assurance in Education literature, and its teaching/learning processes, the assumption is that internal customers' perceptions of this environment directly influence their learning, and the organizational “environment” or “climate” passes to contemplate any learning experience that influences employees' motivation to learn, thereby affecting their attitudes, values, and behaviors related to a learning task (Fernandes et al., 2022a; Newton, 2017).

4.4. Characterization of the parties involved and process of generation of information and demands

The information that runs through a vertical flow from top management to the customer corresponds to measurable indicators unfolded, following specific characteristics of each customer. As a means of transmission, the information is directed to the defined support team, which must communicate and, in some cases, develop the demand together with the service area. To ensure adherence to orders, deliveries are monitored at certain intervals, and through the results of the indicators, customers are measured.

Thus, considering the concept of Quality adopted (International Organization for Standardization, 2015), given by customer satisfaction regarding the suitability of use, it is therefore necessary to structure this process of information generation and demands, as well as to understand the responsibilities of the parties involved. A priori, we must:

• Senior management: is responsible for providing the set of elements, subsidies, or measurable indicators deployed from the company's strategic objectives, through an integrated management model, and that can be applied in specific sectors of operation. The products of senior management are the input information for the IRE and are called resources to be transformed;

• Support team: aiming to strengthen the culture within the company, is responsible for transmitting these resources, through products and support services, aiming to subsidize and provide compliance with product quality specifications and annual production goals. Thus, by transmitting and transforming information, the support team plays an educator/facilitator role while the client is the role of an apprentice, thus configuring a teaching/learning process;

• Operation team: priority, the main responsibility is to meet the goals established by senior management. However, given that the management model must include in integrated way various interests of the organization, the operation must understand the need to add management activities in contrast to operational activities that, in turn, aim to meet the global interests of the organization. In this way, the IRE becomes responsible for informing its demands, needs, and difficulties for a better generation of the products offered.

Thus, since the relationship between internal clients constitutes an internal relationship environment (IRE), the process of generating the information and demands of the IRE is Figure 1 and aims to describe the apparent relationships between the support team and the operation team, as well as the adjacent relationships with the management model and organizational performance.

Figure 1
Diagram of the information and demands generation process.

When observing that the interactions that occur in the IRE contain a feedback loop, either by the change of goals and specifications or by the difference in demands and needs of the operation team. The latter has cyclical and dynamic characteristics associated with the support/application process. For example, the process starts with offering a product/service according to a priority need. When applying, executing, or initiating a new management procedure, it is expected that new needs will emerge and guide the generation of new services.

4.5. Multiple dimensions of quality

In view of the above, we have that the total evaluation of the quality of the model presented, can be divided into four dimensions and Figure 1, namely: [1] quality of transformers, which are all the resources, tools and methods necessary to transform the specific indicators by area, arising from the unfolding of the strategy and goals, into products, services, training and information so that the support team understands its role, know the activities to be carried out, meet the established goals; [2] perceived quality of services, which refers to internal customer satisfaction, and composes the set of products, services, training and information generated as support to the operation; [3] operational quality that, in addition to being measured by meeting the established production goals, is evaluated by the ability of the operation team to manage and achieve the indicators established by the unfolding; [4] quality of the environment, which is the ability in which support teams can support the operation teams, in the sense of effectively transmitting the unfolded information. The latter is characterized both from the point of view of the offeror, by offering the appropriate services to the purpose, as well as by the internal customer, by providing information on demands, needs, and difficulties for better construction of the products offered. Thus, together, these four dimensions should guide the generation of satisfactory results that, because of interaction with distinct types of people, will be aligned with the company's objectives.

Typically, many researchers tend to subdivide perceived quality into various dimensions when considering the different contexts, applications, themes, and subjects involved in the research (Cronin Junior & Taylor, 1994; Petrick, 2002). Although the dimensions of quality are a proposition in this work, the conceptualization was based on a literature review and, although no configuration was found in this format, the search considered the general aspects associated with Quality, perceived quality, perceived quality dimensions, quality management practices, quality assessment, quality measurement, customer satisfaction, perceived value, among others.

Finally, the quality dimensions of the transformers are directly related to both the structure and formulation of the management model, which adequately integrates multiple management tools aiming at organizational excellence, as well as the ability to implement and effectively use the support elements (Bernardo et al., 2022).

4.6. Operationalization of the measurement instrument

Given the validation of the IRE model, and after a detailed description of the impact factors, it is necessary to create, adapt and validate the items. Then, based on the works of (Fernandes et al., 2022b; Fernandes et al., 2022a; Nickel et al., 2010), the procedure for operationalization of the measurement instrument is described in detail below:

• Step 01: Initial elaboration of the items: based on the thematic analysis of content in all information collected previously, this process refers to the identification of the requirements of the clients, that is, it is related to the collection of the needs of characterization of the multiple perspectives, factors, and latent trait;

• Step 02: Grouping and classification of items: refers to the distribution of needs in primary elements of factor specification, that is, the process of directing the items to the respective factor. When there is a larger set of items presented, grouping, and classifying such needs is useful for verifying equivalent items, eliminating repetitions, and the relevant needs;

• Step 03: Definition and hierarchization of factors: refer to the understanding of the researchers of this study regarding the criteria of clarity, relevance, representativeness, and scope, called internal validation. Thus, through a tree structure, it is possible to determine the requirements necessary for the factors to have a deeper and more accurate understanding of what should be of concern in the context in question;

• Step 04: Target question and conversion of items: as a simple way of explanation for intuitive items, it is necessary that when a satisfactory answer to a difficult question is not quickly found, it is necessary to establish a target question that is the evaluation that the research tensions to produce and turn it into a question of adequacy (or empirical) that, in addition to framing the category of answers, it is a simpler question that is answered in place of the target question. The process of converting items, after grouping and sorting, gets clearer and simpler descriptions, that is, measurable questions or expressions. In addition, this conversion consists of a set of objectives, each of which can be seen to achieve an end or the cause of an effect.

At the end of this activity, there is a pilot instrument that can be applied to all operation teams, ready to have their properties evaluated and, in the context of this work, being the practical objective achieved.

5. Results and discussions

5.1. Conceptual Model of the Internal Relationship Environment (IRE)

Considering that the conceptual model aims to relate the IRE with the level of satisfaction of the operational teams, the construction of a measurement instrument constitutes a property to evaluate the perceived quality and, specifically, the object of this study is the latent trait “differences in perceived quality operations team members concerning the internal relationship environment (IRE)”.

Thus, given the latent trait, it can be measured by the satisfaction attribute, considering the several factors that impact the IRE. The hypothesis is that there is a dominant factor among the parties involved that manifests itself through the exchange of information (feedback) necessary for support. The conceptual model that exemplifies this relationship is presented in Figure 2 and the impact factors are poor communication, horizontal service, service provision, and andragogy. In addition, to obtain a self-descriptive statement from the responses, we will be able to discover the distinct levels of intensity of opinion regarding customer service quality. Statements are presented to respond with a degree of agreement with that sentence. For this, customers must mark, on the scale, the answer that best reflects their opinion, such as 1) Very dissatisfied, 2) Dissatisfied, 3) Neutral, 4) Satisfied and 5) Very satisfied.

Figure 2
Conceptual Model of the Internal Relationship Environment (IRE).

The presentation and description of each factor follow:

• Factor 1: Poor Communication: Refers to poor communication of the information generated, which in turn results in products inconsistent with customer demands. If the transmitter, which is the element that issues the message to the receiver, has noise, interfering or compromising the transmission of the message, through the communication channel used, it will impact the deviation of the product to the proposed interests. These failures may result from a poor definition of product specifications, a delay or misunderstanding of demand, or even the procedures and tools used for communication;

• Factor 2: Horizontalized service: It refers to customer service in an egalitarian way, not prioritizing the manager (Manager, Supervisor, Coordinator), but understanding the desires and needs of employees on the shop floor. In this way, we can work more effectively in solving problems and, at the same time, value the service and/or activity of each person at the end of the operation;

• Factor 3: Service offer: Refers to the form of presentation of the demands to be executed. Thus, assuming that, if the offer is not presented to the customer in a correct and uncomplicated way, the declaration of results and products obtained will be different from the proposed objective. This can cause embarrassment in the internal relationship environment;

• Factor 4: Andragogy: It is related to adult education, respecting independence and self-management in their environment, in addition to clarifying the importance and applicability of this alignment in their daily lives. In this way, a prospective acceptance of the proposed demands is expected, as an understanding at all hierarchical levels, motivation in the execution of mandatory procedures, and, consequently, minimizing the rejection of the services offered.

The concept of satisfaction or perception of quality in this work refers to the idea of a search for criteria that measure the similarity between knowledge and the object that corresponds to it. Therefore, considering the internal relationship environment (IRE), in which employees are inserted, and incorporating aspects of organizational and social integration, organizational culture, people development, standardization of best practices, operational discipline, and teaching/learning process, the hypothesis is that there is a set of factors that exert direct influence in this environment and that, in turn, generates a negative, neutral or positive perception associated with the total quality of this environment. From the perspective of the internal client, the focus of this study, the impact of these factors will generate dissatisfaction, neutrality, or satisfaction with the IRE, which is managed by the support team.

By treating this process from the perspective of teaching/learning, in which the perceptions of internal clients about this environment directly influence their learning, the organizational “environment” or “climate” is any learning experience that influences the motivation of employees to learn, affecting their attitudes, values, and behaviors related to a learning task (Fernandes et al., 2022b; Newton, 2017). Thus, given the dimensions of quality presented, it is possible to relate them to the diverse works in the fields of behavioral psychology, psychometrics, and education, in which they provide a framework of methods, tools, theories, and models to explain and describe phenomena, which, which are related to the subject of interest, can serve as a basis for the creation of constructs that are manifestations of reality observed indirectly through other variables that can be observed (De Ayala, 2013; Pasquali, 2017a; Zanon et al., 2016).

The Environment Quality dimension assumes that the change in the method of transmission of information can cause differences in the quality perceived by internal customers, that is, the need of each operation team results in different services offered and, in turn, results in different responses. From a theoretical point of view, this phenomenon can be explained by the S-O-R theory which originally suggests that environmental stimuli (S) induce an emotional reaction (O) and subsequently result in a behavioral response (R) (Mehrabian & Russell, 1974) Correlated with the proposed conceptual model, it is expected that the factors that impact the environment (S) induce satisfaction (O) and, in turn, provide a better operational discipline focused on operational excellence (R).

5.2. Identification of customer requirements

As a way of providing additional support, the feasibility of the construct with content validity, according to (Pasquali, 2017b), requires the specification before the construction of the items, requiring the definition of what to measure, the explanation of the processes to be evaluated and the determination of the relative proportion of representation in the test of each content topic. In the case of this study, the definition constitutes the latent trait already determined, the explanation of the processes refers to the resulting conceptual products and, finally, the determination of the relative proportion of the categories was achieved in the two focus groups (exploratory and confirmatory), where the evaluation was made by comparing pairs of factors, as well as determining how much each one is more or less important for the construct.

Given the determination and description of the factors that impact the IRE, it was possible to list the most relevant elements, and these represent aspects of each factor, derived from the thematic content analysis, and treated as requirements of customer interest. The definition of the construct's requirements and the minimum requirements that each factor must have been presented in Figure 3. Note that the requirements are a direct derivation of the resulting products and constitute relevant aspects of an internal relationship environment.

Figure 3
Identification diagram of customer requirements based on factors.

The excess of information, requires a comprehensive and integrated management system, considering the work environment, machinery, and the human factor, these three perspectives being a decision-making factor in the performance of the process and raising their cross-sections, health issues, safety, Environment, communities and risks, quality, productivity and costs (Skrzypkowski et al., 2020); [2] Abstention from management leadership, and involves elements such as negligence, absence of a systemic vision and an integrated process, conflict of interests between teams or a particular method of work (Bernardo et al., 2022), and finally; [3] People management, which according to (Xu et al., 2020), refers to the extent to which employees engage in management, including practices such as training, employee relations, capacity building, workforce management work, teamwork, and people involvement

5.3. Operationalization of the measuring instrument

This procedure represents a hierarchical structure of factors, requirements, and items, each one containing a specific measurement purpose, so that the process of adoption, implementation, and execution of the IMS, has the function of establishing the monitoring and control process, demonstrating operational excellence and be the reference for assessing the quality of operating processes.

At the top of the hierarchy is the latent trait, the most aggregate level of the data. The two-level is composed of primary factors, each of which consists of several attributes of interest. Once the most relevant aspects have been identified, which are treated as requirements of evaluation interest, it is necessary to convert these needs into indicators, that is, transform them into items with measurement purposes that must be measured. The set of items is the evaluation that the instrument intends to produce and that will be manifested by the level of satisfaction.

Table 1 contemplates the operational procedure of the measurement instrument. The tree structure helped to clarify the decision context, validation, and determination of representativeness for each content topic, that is, it becomes possible to assess whether the item meets the customer's requirements and compose factors. Thus, this work, in addition to understanding the potential construction biases of the items, initially provided a set of four factors, twelve customer requirements, and forty items or requirements for evaluation.

Table 1
Measuring Instrument.

ITEMS	EVALUATION INTEREST REQUIREMENTS	PURPOSE OF MEASUREMENT	ITEM DESCRIPTION
F01_01	COMUNICATION VIA EMAIL	Communication Agility	As for the agility of communication via email with the support team, how do you feel?
F01_02	COMUNICATION VIA EMAIL	Number of emails	As for the number of emails exchanged with the support team, how do you feel?
F01_03	COMUNICATION VIA EMAIL	Information exchange	As for the ease of exchanging information, before formalizing it by e-mail, with the support team, how do you feel?
F01_04	COMMUNICATION TO LEADERS	Delivery deadlines (Clarity Criterion)	As for the clarificationto shift leaders when establishing deadlines for delivery of the requested demands, how do you feel?
F01_05	COMMUNICATION TO LEADERS	Relevance of claims (Representativeness criteria)	As for the pertinence/relevance of the requested demands to shift leaders, how do you feel?
F01_06	COMMUNICATION TO LEADERS	Coverage of demands (Coverage criterion)	As for the clarification made to shift leaders on the coverage of the requests requested, how do you feel? (Do the demands cover all aspects of interest to the operation?)
F01_07	EXCESS INFORMATION	volume of Information	As for the volume of information sent by the support team, how do you feel?
F01_08	EXCESS INFORMATION	Understanding of Information	As for the Comprehension of all information (collectively) exchanged with the support team, how do you feel? (Is the excess of information confusing/difficult?)
F01_09	EXCESS INFORMATION	Information management flexibility (Coverage criterion)	As for the flexibility in managing excess information sent by the support team, how do you feel? (Time Flexibility = Decide when to do it and do it at your own pace)
F01_10	EXCESS INFORMATION	Information content flexibility (Coverage criterion)	As for the flexibility of choice over excess information send by the support team, how do you feel? (Time Flexibility = Decide when to do it and do it at your own pace)
F02_01	ATTENTION TO ALL OF THE TEAM	Service quality	As for the service quality of the support team in serving all employees up to the shop floor level, how do you feel?
F02_02	ATTENTION TO ALL OF THE TEAM	Preparation and knowledge for service	As for the preparation and knowledge for service of the support team in serving all employees up to the shop floor level, how do you feel?
F02_03	ATTENTION TO ALL OF THE TEAM	Approach to service	As for the approach to service of the support team in serving all employees up to the shop floor level, how do you feel?
F02_04	ENGAGEMENT OF EMPLOYEES	Problems solution	As for the engagement of all employees for problems solution within their areas, how do you feel?
F02_05	ENGAGEMENT OF EMPLOYEES	Development in continuous improvement programs	As for the engagement of all employees for development in continuous improvement programs within their areas, how do you feel?
F02_06	ENGAGEMENT OF EMPLOYEES	Compliance with process procedures	As for the engagement of all employees to comply with process procedures within their areas, how do you feel?
F02_07	ENGAGEMENT OF EMPLOYEES	The self-regulated effort with tasks	As for the determination/dedication of all employees in dealing with management activities considered difficult within their areas, how do you feel?
F02_08	SCHEDULE WITH SHOP FLOOR LEVEL	Frequency of schedules	As for the frequency of the support team's agendas with all employees up to the shop floor level, how do you feel?
F02_09	SCHEDULE WITH SHOP FLOOR LEVEL	Getting feedback (Clarity Criterion)	As for getting feedback on the support team's agendas with all employees up to the shop floor level, how do you feel? (Is there a clear concern to hear the reports of all employees?)
F02_10	SCHEDULE WITH SHOP FLOOR LEVEL	Purpose of agendas (Clarity Criterion)	As for the clarity of the purposes of the support team's agendas with all employees up to the shop floor level, how do you feel? (Are the purposes of the agenda clarified?)
F02_11	SCHEDULE WITH SHOP FLOOR LEVEL	Relevance of agendas (Representativeness criteria)	As for the Relevance of the support team's agendas with all employees up to the shop floor level, how do you feel? (Are the agendas relevant and do they achieve the proposed objectives?)
F02_12	SCHEDULE WITH SHOP FLOOR LEVEL	Number of topics (Coverage criterion)	As for the number of topics/issues discussed in the support team's agendas for all employees up to the shop floor level, how do you feel?
F03_01	SERVICE PRESENTATION FORM	Clarity of the presentation (Clarity Criterion)	As for the clarity of the presentation of the services offered by the support team, how do you feel? (Does the presentation allow you to understand the service offered?)
F03_02	SERVICE PRESENTATION FORM	Representativeness of the presentation (Representativeness criteria)	As for the representativeness of the presentation of the services presentation support team, how do you feel? (Does the presentation demonstrate the reality of the problem and justify the offer?)
F03_03	SERVICE PRESENTATION FORM	Coverage of the presentation (Coverage criterion)	As for the coverage of the presentation of the services offered by the support team, how do you feel? (Does the presentation cover the complete set of aspects that make up the problem?)
F03_04	SERVICE PRESENTATION FORM	Practical applicability	As for the presentation of the practical applicability offered by the support team, how do you feel? (Does the presentation demonstrate the practical applicability of the services offered?)
F03_05	EXCESS SUPPLY (DEMAND)	deadline between the request	As for the deadline between the request/demand delivery established by the support team, how do you feel?
F03_06	EXCESS SUPPLY (DEMAND)	Information management flexibility (Flexibility criterion)	As for the possibility of flexible delivery times established by the support team, how do you feel?
F03_07	EXCESS SUPPLY (DEMAND)	Number of demands	As for the number of demands requested by the support team, how do you feel?
F03_08	EXCESS SUPPLY (DEMAND)	Accumulation of demands	As for the accumulation of old demands x new demands requested by the support team, how do you feel?
F03_09	EXCESS SUPPLY (DEMAND)	Hierarchy of demands (Coverage criterion)	As for the support in prioritizing the excess demands requested by the support team, how do you feel?
F03_10	SPACE FOR DOUBT ABOUT DEMAND	Current schedule	As for the current schedule stipulated by the support team, com focus on clearing doubts about the demands, how do you feel?
F03_11	SPACE FOR DOUBT ABOUT DEMAND	Atendimento de Dúvidas (Flexibility criterion)	As for the flexibility for service for the support team, focusing on clarifying doubts about the demands, how do you feel? (Time Flexibility = Decide when to do it and do it at your own pace)
F04_01	UNDERSTANDING AT ALL LEVELS	Model clarification (Clarity Criterion)	As for the clarity of the demand’s presentation model, that is, if it can be well understood by all employees up to the shop floor level, how do you feel?
F04_02	UNDERSTANDING AT ALL LEVELS	Model relevance (Representativeness criteria)	As for the relevance of the presentation model, that is, if the content contained is representative and suitable for all employees up to the shop floor level, how do you feel?
F04_03	UNDERSTANDING AT ALL LEVELS	Model Coverage (Coverage criterion)	As for the coverage of the presentation model, that is, if the content contained covers the set of characteristics of all employees up to the shop floor level, how do you feel?
F04_04	UNDERSTANDING AT ALL LEVELS	Difficulty of understanding	As for the difficulties that all employees up to the shop floor level to understand the purpose of the process they conduct, how do you feel?
F04_05	SCIENCE OF THE IMPORTANCE OF DELIVERY		As for the motivation to conduct the mandatory procedures, developed by the shop floor, how do you feel?
F04_06	SCIENCE OF THE IMPORTANCE OF DELIVERY		As for the difficulties, which all employees up to the shop floor level, to understand the purpose of the process they conduct, how do you feel?

It is observed that the flow begins with the determination of factors, identification of customer requirements, appropriation of requirements to factors, and, finally, identification of assessment requirements. The assessment requirements construction flow was based on the multicriteria model for the localized informational design phase of product development (Robertson, 1996) and, specifically for this flow, consists of converting customer requirements into measurable expressions. through mind maps in chains of means and ends. Ends the more, these expressions are related to response scales.

5.4. The validity procedure

The validity, according to (Pasquali, 2017b, p. 14): “[...] occupies a central position in measurement theory, constituting a fundamental and indispensable parameter, defined as the extent to which empirical evidence supports the interpretations and proposed uses for the test”. In this perspective, the first validation procedure is theoretical, and described in section 2 - Theoretical Foundation, and, in turn, addresses the two main concepts addressed in this research, the integrated management system (IMS) and the evaluation process. of the quality of the IMS.

A second validation procedure, also theoretical, is the process of systematic literature review and is the central resource in research for the development of measurement instruments (Coluci et al., 2015). Therefore, the validation consists of the description of the Measuring Organizational Performance which, as discussed in section 3, allows understanding: [1] the evaluations of the processes by comparing the practical performance (actual) versus the programmed goal (estimated), scaled by KPIs, [2] the implementation and execution strategy of the IMS itself, which determines the maturity assessment process, [3] the formal quality assessment processes (QA) and, [4] the Perceived Quality Assessment Process. In this way, and incorporating the mining context, this validation made it possible to understand the concept of Operational Excellence, in the sense of exceptional quality, and transcends the concepts of operational effectiveness and efficiency.

Finally, a third validation procedure refers to empirical and practical application. Therefore, this characterization is described in this section, in which it describes the application context, the data analysis, and collection procedure, presents the conceptual model of the Internal Relationship Environment, and presents the characterization of the parties involved and the information generation process.

6. Final considerations

With a focus on evaluating how information travels vertically from top management to the operation team and, consequently, results in an efficient measurement of operational performance, it can be said that the main objective of this work was achieved by operationalizing the measurement instrument this one, which meets the three test validation procedures.

The answers to (RQ₁) and (RQ₂) jointly represent the three validation procedures of the test, which, in another way, is about the alignment between the theoretical foundation, organizational performance measurement, and mining context. In summary, for (RQ₁), the dissemination strategy directly affects operational performance which, in addition to meeting the goals given by the KPIs, incorporates best management practices and coincides with the collective considerations of the effects of individual practices on the dimensions of organizational performance and improvement continuous (Xu et al., 2020).

Otherwise, (RQ₂), is a direct application of the company under study which, as a justification, it is assumed that an operation team with a lower level of maturity will have greater difficulties in adopting best practices and, consequently, will tend to have unsatisfactory results, that is, non-compliance of production targets. Therefore, it is reasonable to think that the maturity of each team tends to affect operational performance, which is an indirect form of measurement.

Nevertheless, in addition to a bibliographic review, the realization of an exploratory and confirmatory analysis of impact factors, aspects of interest to the organization, and responsibilities of the parties involved answers the third research question (RQ₃). Therefore, because of this exploration and confirmation, we obtained the theoretical and practical foundations that validate the conceptual model when describing the internal relationship environment (IRE) and which incorporates the differences in perceived quality between the parties.

Finally, to respond to (RQ₄), the team's perception is expected to directly affect, and to a considerable extent, the IMS implementation process. However, this is an empirical validation based on little information. Therefore, the gold standard for test validation is achieved after planning, application, collection, and processing of data and, later, by statistical analysis and internal consistency and is not the focus of this work (Pasquali, 2017b).

Thus, evaluation of the total quality of the IRE will emphasize the end-activity (achievement of operational excellence by the operation team), given through the activity-beginning (metrics deployed from the strategic objectives by senior management), thus considering the consequences of the middle activities (the IRE and impact factors) managed by a support team.

• How to cite this article: Fernandes, R. S., Rocha, T. R., Coelho, J. M., & Andrade, D. F. (2023). Development of a measurement instrument to evaluate integrated management systems and differences in perception: an approach to item response theory and the quality management process. Production, 33, e20220069. https://doi.org/10.1590/0103-6513.20220069

References

• Andrade, D. F., Tavares, H. R., & da Cunha Valle, R. (2000). Teoria da Resposta ao Item: conceitos e aplicações Sao Paulo: ABE.
• Asif, M., Searcy, C., Zutshi, A., & Ahmad, N. (2011). An integrated management systems approach to corporate sustainability. European Business Review, 23(4), 353-367. http://dx.doi.org/10.1108/09555341111145744
  » http://dx.doi.org/10.1108/09555341111145744
• Banta, T. W., Pike, G. R., & Hansen, M. J. (2009). The use of engagement data in accreditation, planning, and assessment. New Directions for Institutional Research, 2009(141), 21-34. http://dx.doi.org/10.1002/ir.284
  » http://dx.doi.org/10.1002/ir.284
• Barratt, T., & Ellem, B. (2019). Temporality and the evolution of GPNs: remaking BHP’s Pilbara iron ore network. Regional Studies, 53(11), 1555-1564. http://dx.doi.org/10.1080/00343404.2019.1590542
  » http://dx.doi.org/10.1080/00343404.2019.1590542
• Basso, V., Andrade, B., Jacovine, L., Silva, E., Alves, R., & Nardelli, A. (2020). Forest management certification in the Americas: difficulties in complying with the requirements of the FSC system. International Forestry Review, 22(2), 169-188. http://dx.doi.org/10.1505/146554820829403478
  » http://dx.doi.org/10.1505/146554820829403478
• Battilana, J., Obloj, T., Pache, A.-C., & Sengul, M. (2020). Beyond shareholder value maximization: accounting for financial/social trade-offs in dual-purpose companies. Academy of Management Review, 47(2), 237-258. http://dx.doi.org/10.5465/amr.2019.0386
  » http://dx.doi.org/10.5465/amr.2019.0386
• Belotti Pedroso, C., Tate, W. L., Lago da Silva, A., & Ribeiro Carpinetti, L. C. (2021). Supplier development adoption: a conceptual model for triple bottom line (TBL) outcomes. Journal of Cleaner Production, 314, 127886. http://dx.doi.org/10.1016/j.jclepro.2021.127886
  » http://dx.doi.org/10.1016/j.jclepro.2021.127886
• Bernardo, S. M., Rampasso, I. S., Quelhas, O. L., Leal Filho, W., & Anholon, R. (2022). Method to integrate management tools aiming for organizational excellence. Production, 32, e20210101. http://dx.doi.org/10.1590/0103-6513.20210101
  » http://dx.doi.org/10.1590/0103-6513.20210101
• Bhadani, K., Asbjörnsson, G., Hulthén, E., & Evertsson, M. (2020). Development and implementation of key performance indicators for aggregate production using dynamic simulation. Minerals Engineering, 145, 106065. http://dx.doi.org/10.1016/j.mineng.2019.106065
  » http://dx.doi.org/10.1016/j.mineng.2019.106065
• Bokrantz, J., Skoogh, A., Berlin, C., Wuest, T., & Stahre, J. (2020). Smart Maintenance: a research agenda for industrial maintenance management. International Journal of Production Economics, 224, 107547. http://dx.doi.org/10.1016/j.ijpe.2019.107547
  » http://dx.doi.org/10.1016/j.ijpe.2019.107547
• Bouslah, B., Gharbi, A., & Pellerin, R. (2016). Integrated production, sampling quality control, and maintenance of deteriorating production systems with AOQL constraints. Omega, 61, 110-126. http://dx.doi.org/10.1016/j.omega.2015.07.012
  » http://dx.doi.org/10.1016/j.omega.2015.07.012
• Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77-101. http://dx.doi.org/10.1191/1478088706qp063oa
  » http://dx.doi.org/10.1191/1478088706qp063oa
• Bulkan, J. (2020). Smallholder forestry in the FSC system: a review. Revue Gouvernance/Governance Review, 17(2), 7-29. https://doi.org/10.7202/1073109ar
  » https://doi.org/10.7202/1073109ar
• Chatterjee, S., Narayanan, V., & Malek, W. (2016). How strategy execution maps guided Cisco System’s Sales Incentive Compensation plan. Strategy and Leadership, 44(6), 25-34. http://dx.doi.org/10.1108/SL-08-2016-0071
  » http://dx.doi.org/10.1108/SL-08-2016-0071
• Coluci, M. Z. O., Alexandre, N. M. C., & Milani, D. (2015). Construção de instrumentos de medida na área da saúde. Ciencia & Saude Coletiva, 20(3), 925-936. http://dx.doi.org/10.1590/1413-81232015203.04332013 PMid:25760132.
  » http://dx.doi.org/10.1590/1413-81232015203.04332013
• Cronin Junior, J. J., & Taylor, S. A. (1994). Server versus servqual: reconciling performance-based and perceptions-minus-expectations measurement of service quality. Journal of Marketing, 58(1), 125-131. http://dx.doi.org/10.1177/002224299405800110
  » http://dx.doi.org/10.1177/002224299405800110
• De Ayala, R. J. (2013). The theory and practice of item response theory New York: Guilford Publications.
• Drummond, P., Araujo, F., & Borges, R. (2017). Meeting halfway: assessing the differences between the perceptions of ERP implementers and end-users. Business Process Management Journal, 23(5), 936-956. http://dx.doi.org/10.1108/BPMJ-05-2016-0107
  » http://dx.doi.org/10.1108/BPMJ-05-2016-0107
• Elassy, N. (2015). The concepts of quality, quality assurance, and quality enhancement. Quality Assurance in Education, 23(3), 250-261. http://dx.doi.org/10.1108/QAE-11-2012-0046
  » http://dx.doi.org/10.1108/QAE-11-2012-0046
• Fernandes, R. S., Biffe, B. G., Louzada, M. J. Q., Bornia, A. C., & Andrade, D. F. (2022a). Standardized measure for performance assessment of athletes in the crossfit open: theoretical structuring and item response theory. Research Square, 1-15. https://doi.org/10.21203/rs.3.rs-1308148/v1
  » https://doi.org/10.21203/rs.3.rs-1308148/v1
• Fernandes, R. S., Luz, R. M. N., Reis, D. C., Luz, M. A. L., & Guimarães, G. V. (2022b). Elaboration of quality perception instrument of remote teaching amidst COVID-19 pandemics in a University of Northern Brazil. Research Square, 1-20. https://doi.org/10.21203/rs.3.rs-1308160/v1
  » https://doi.org/10.21203/rs.3.rs-1308160/v1
• Fernandes, R. S., Sousa, L. R. C., & Santos, T. L. (2021). Análise, investigação e avaliação da gestão da qualidade no processo de mineração. Revista Produção Online, 21(3), 770-793. http://dx.doi.org/10.14488/1676-1901.v21i3.4252
  » http://dx.doi.org/10.14488/1676-1901.v21i3.4252
• Gackowiec, P., Podobińska-Staniec, M., Brzychczy, E., Kühlbach, C., & Özver, T. (2020). Review of key performance indicators for process monitoring in the mining industry. Energies, 13(19), 5169. http://dx.doi.org/10.3390/en13195169
  » http://dx.doi.org/10.3390/en13195169
• Green, D. (1994). What Is quality in higher education? London: ERIC.
• Hadidi, L. A., Turki, U. M. A., & Rahim, A. (2012). Integrated models in production planning and scheduling, maintenance and quality: a review. International Journal of Industrial and Systems Engineering, 10(1), 21-50. http://dx.doi.org/10.1504/IJISE.2012.044042
  » http://dx.doi.org/10.1504/IJISE.2012.044042
• Hezri, A. A., & Dovers, S. R. (2006). Sustainability indicators, policy, and governance: Issues for ecological economics. Ecological Economics, 60(1), 86-99. http://dx.doi.org/10.1016/j.ecolecon.2005.11.019
  » http://dx.doi.org/10.1016/j.ecolecon.2005.11.019
• Houaiss, A. (2001). Dicionário eletrônico Houaiss da língua portuguesa Rio de Janeiro: Ed. Objetiva.
• International Organization for Standardization – ISO. (2014a). ISO 22400-1. Automation systems and integration — Key performance indicators (KPIs) for manufacturing operations management — Part 1: Overview, concepts, and terminology (pp. 19). Geneva, Switzerland: ISO.
• International Organization for Standardization – ISO. (2014b). ISO 22400-2. Automation Systems and Integration - Key Performance Indicators (KPIs) for Manufacturing Operations Management, Part 2: Definitions and Descriptions Geneva, Switzerland: ISO.
• International Organization for Standardization – ISO. (2015). ISO 9001. Quality management systems - Fundamentals and vocabulary (pp. 29). Geneva, Switzerland: ISO.
• International Organization for Standardization – ISO. (2017). ISO 22400-2/AMD 1. Automation systems and integration — Key performance indicators (KPIs) for manufacturing operations management — Part 2: Definitions and descriptions — Amendment 1: Key performance indicators for energy management (pp. 10). Geneva, Switzerland: ISO.
• International Organization for Standardization – ISO. (2018). ISO/TR 22400-10. Automation systems and integration — Key performance indicators (KPIs) for manufacturing operations management — Part 10: Operational sequence description of data acquisition (pp. 34). Geneva, Switzerland: ISO.
• International Organization for Standardization – ISO. (2020). ISO 9000. Quality management systems - Fundamentals and vocabulary (pp. 23). Geneva, Switzerland: ISO.
• International Organization for Standardization – ISO. (2021). ISO 10013. Quality management systems. In Guidance for documented information (pp. 14). Geneva, Switzerland: ISO.
• Juran, J. M. (1992). Juran on quality by design: the new steps for planning quality into goods and services New York: Simon and Schuster.
• Kang, N., Zhao, C., Li, J., & Horst, J. A. (2016). A Hierarchical structure of key performance indicators for operation management and continuous improvement in production systems. International Journal of Production Research, 54(21), 6333-6350. http://dx.doi.org/10.1080/00207543.2015.1136082 PMid:29398722.
  » http://dx.doi.org/10.1080/00207543.2015.1136082
• King, L. O. (2016). Functional sustainability indicators. Ecological Indicators, 66, 121-131. http://dx.doi.org/10.1016/j.ecolind.2016.01.027
  » http://dx.doi.org/10.1016/j.ecolind.2016.01.027
• Lee, R. G., & Dale, B. G. (1998). Business process management: a review and evaluation. Business Process Management Journal, 4(3), 214-225. http://dx.doi.org/10.1108/14637159810224322
  » http://dx.doi.org/10.1108/14637159810224322
• Maheswari, C., Priyanka, E., Thangavel, S., Vignesh, S. R., & Poongodi, C. (2020). Multiple regression analysis for the prediction of extraction efficiency in the mining industry with industrial IoT. Production Engineering, 14(4), 457-471. http://dx.doi.org/10.1007/s11740-020-00970-z
  » http://dx.doi.org/10.1007/s11740-020-00970-z
• Mehrabian, A., & Russell, J. A. (1974). An approach to environmental psychology Cambridge: The MIT Press.
• Miles, M. P., & Covin, J. G. (2000). Environmental marketing: a source of reputational, competitive, and financial advantage. Journal of Business Ethics, 23(3), 299-311. http://dx.doi.org/10.1023/A:1006214509281
  » http://dx.doi.org/10.1023/A:1006214509281
• Mohammadi, M., Rai, P., & Gupta, S. (2017). Performance evaluation of bucket-based excavating, loading, and transport (BELT) equipment–an OEE approach. Archives of Mining Sciences, 62(1), 105-120. http://dx.doi.org/10.1515/amsc-2017-0008
  » http://dx.doi.org/10.1515/amsc-2017-0008
• Mohsin, M., Zhu, Q., Naseem, S., Sarfraz, M., & Ivascu, L. (2021). Mining industry impact on environmental sustainability, economic growth, social interaction, and public health: an application of semi-quantitative mathematical approach. Processes (Basel, Switzerland), 9(6), 972. http://dx.doi.org/10.3390/pr9060972
  » http://dx.doi.org/10.3390/pr9060972
• Nairn, R. W., LaBar, J. A., Oxenford, L. R., Shepherd, N. L., Holzbauer-Schweitzer, B. K., Arango, J. G., Tang, Z., Dorman, D. M., Folz, C. A., & McCann, J. I. (2020). Toward sustainability of passive treatment in legacy mining watersheds: operational performance and system maintenance. In Proceedings of the IMWA 2020 “Mine Water Solutions (pp. 123-128). Christchurch, New Zealand: IMWA.
• Newton, P. E. (2017). There is more to educational measurement than measuring: the importance of embracing purpose pluralism. Educational Measurement: Issues and Practice, 36(2), 5-15. https://doi.org/10.1111/emip.12146
  » https://doi.org/10.1111/emip.12146
• Nicholls, J. A. (2020). Integrating financial, social, and environmental accounting. Sustainability Accounting, Management and Policy Journal, 11(4), 745-769. http://dx.doi.org/10.1108/SAMPJ-01-2019-0030
  » http://dx.doi.org/10.1108/SAMPJ-01-2019-0030
• Nickel, E. M., Ferreira, M. G. G., Forcellini, F. A., Santos, C. T., & Silva, R. A. Á. (2010). Modelo multicritério para referência na fase de Projeto Informacional do Processo de Desenvolvimento de Produtos. Gestão & Produção, 17(4), 707-720. http://dx.doi.org/10.1590/S0104-530X2010000400006
  » http://dx.doi.org/10.1590/S0104-530X2010000400006
• Nunes, R. A., Delboni Junior, H., Tomi, G., Infante, C. B., & Allan, B. (2019). A decision-making method to assess the benefits of a semi-mobile in-pit crushing and conveying alternatives during the early stages of a mining project. REM - International Engineering Journal, 72(2), 285-291. http://dx.doi.org/10.1590/0370-44672018720109
  » http://dx.doi.org/10.1590/0370-44672018720109
• Ory, J. C. (1992). Meta-assessment: evaluating assessment activities. Research in Higher Education, 33(4), 467-481. http://dx.doi.org/10.1007/BF00973767
  » http://dx.doi.org/10.1007/BF00973767
• Otto, T., & Musingwini, C. (2020). A compliance driver tree (CDT) based approach for improving the alignment of spatial and intertemporal execution with mine planning at open-pit mines. Resources Policy, 69, 101826. http://dx.doi.org/10.1016/j.resourpol.2020.101826
  » http://dx.doi.org/10.1016/j.resourpol.2020.101826
• Pan, X., Sinha, P., & Chen, X. (2021). Corporate social responsibility and eco-innovation: the triple bottom line perspective. Corporate Social Responsibility and Environmental Management, 28(1), 214-228. http://dx.doi.org/10.1002/csr.2043
  » http://dx.doi.org/10.1002/csr.2043
• Parmenter, D. (2015). Key performance indicators: developing, implementing, and using winning KPIs Hoboken: John Wiley & Sons.
• Pascual, R., Madariaga, R., Santelices, G., Godoy, D., & Droguett, E. L. (2016). A structured methodology to optimize the throughput of production lines. International Journal of Mining, Reclamation and Environment, 30(1), 25-36. http://dx.doi.org/10.1080/17480930.2014.962235
  » http://dx.doi.org/10.1080/17480930.2014.962235
• Pasquali, L. (2017a). Psicometria: teoria dos testes na psicologia e na educação São Paulo: Editora Vozes Limitada. Retrieved in 2022, June 1, from https://examen.emnuvens.com.br/rev/article/view/19
  » https://examen.emnuvens.com.br/rev/article/view/19
• Pasquali, L. (2017b). Validade dos testes. Examen: Política, Gestão e Avaliação da Educação, 1(1), 14-48. Retrieved in 2022, June 1, from https://examen.emnuvens.com.br/rev/article/view/19
  » https://examen.emnuvens.com.br/rev/article/view/19
• Pasquali, L. (2020). TRI–Teoria de resposta ao item: teoria, procedimentos e aplicações Curitiba: Editora Appris.
• Peral, J., Maté, A., & Marco, M. (2017). Application of data mining techniques to identify relevant key performance indicators. Computer Standards & Interfaces, 54, 76-85. http://dx.doi.org/10.1016/j.csi.2016.11.006
  » http://dx.doi.org/10.1016/j.csi.2016.11.006
• Pereira, L., & Nunes, N. (2018). Performance evaluation in non‐intrusive load monitoring: datasets, metrics, and tools—A review. Wiley Interdisciplinary Reviews. Data Mining and Knowledge Discovery, 8(6), e1265. http://dx.doi.org/10.1002/widm.1265
  » http://dx.doi.org/10.1002/widm.1265
• Petrick, J. F. (2002). Development of a multi-dimensional scale for measuring the perceived value of a service. Journal of Leisure Research, 34(2), 119-134. http://dx.doi.org/10.1080/00222216.2002.11949965
  » http://dx.doi.org/10.1080/00222216.2002.11949965
• PMBOK Guide. (2021). A Guide to the Project Management Body of Knowledge (PMBOK guide) (pp. 763). Pennsylvania: Project Management Institute/Newtown Square.
• Qiao, G., Schlenoff, C., & Weiss, B. A. (2017). Quick positional health assessment for industrial robot prognostics and health management (PHM). In Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA) USA: IEEE. http://dx.doi.org/10.1109/ICRA.2017.7989214
  » http://dx.doi.org/10.1109/ICRA.2017.7989214
• Quigley, M., & Dimitrakopoulos, R. (2020). Incorporating geological and equipment performance uncertainty while optimizing short-term mine production schedules. International Journal of Mining, Reclamation and Environment, 34(5), 362-383. http://dx.doi.org/10.1080/17480930.2019.1658923
  » http://dx.doi.org/10.1080/17480930.2019.1658923
• Rocha, C. S., Cardoso, E. L. S. S., Fernandes, R. S., Branco, N. C. N. M., & Luz, R. M. N. (2021). Quality assessment of divergences caused in weighing copper ore at a mining company in the amazon region. International Journal of Developmental Research, 11(8), 49633-49639. http://dx.doi.org/10.37118/ijdr.22578.08.2021
  » http://dx.doi.org/10.37118/ijdr.22578.08.2021
• Robertson, J. (1996). Review of: “ Product Design Fundamentals and Methods ” by N. F. M. ROOZENBURG and J. EEKELS, Wiley (1995), pp, 397, £27.50, ISBN 0-471-95465-9. Ergonomics, 39(7), 992-993. http://dx.doi.org/10.1080/00140139608964522
  » http://dx.doi.org/10.1080/00140139608964522
• Sane, R. (2018). Beneficiation and agglomeration of manganese ore fines (an area so important and yet so ignored). IOP Conference Series: Materials Science and Engineering, 285, 012033. http://dx.doi.org/10.1088/1757-899X/285/1/012033.
• Santos, M. S., Pinto, T. V., Júnior, Ê. L., Cota, L. P., Souza, M. J., & Euzébio, T. A. (2020). Simheuristic-based decision support system for efficiency improvement of an iron ore crusher circuit. Engineering Applications of Artificial Intelligence, 94, 103789. http://dx.doi.org/10.1016/j.engappai.2020.103789
  » http://dx.doi.org/10.1016/j.engappai.2020.103789
• Shen, X., Chen, L., Xia, S., Xie, Z., & Qin, X. (2018). Burdening proportion and new energy-saving technologies analysis and optimization for iron and steel production system. Journal of Cleaner Production, 172, 2153-2166. http://dx.doi.org/10.1016/j.jclepro.2017.11.204
  » http://dx.doi.org/10.1016/j.jclepro.2017.11.204
• Silva, B. L. F., Santos Neto, I., Fernandes, R. S., Branco, N. C., & Guimarães, G. V. (2021). Dimensionamento e viabilidade econômica de um sistema fotovoltaico: um estudo de caso na UFRA/Parauapebas. Revista Produção Online, 21(3), 863-890. http://dx.doi.org/10.14488/1676-1901.v21i4.4342
  » http://dx.doi.org/10.14488/1676-1901.v21i4.4342
• Singh, R. K., Murty, H. R., Gupta, S. K., & Dikshit, A. K. (2009). An overview of sustainability assessment methodologies. Ecological Indicators, 9(2), 189-212. http://dx.doi.org/10.1016/j.ecolind.2008.05.011
  » http://dx.doi.org/10.1016/j.ecolind.2008.05.011
• Sisodia, R., & Villegas Forero, D. (2019). Quality 4.0 – how to handle quality in the industry 4.0 revolution (Report No. Number E2019:128). Gothenburg, Sweden: Chalmers University of Technology. Retrieved in 2022, June 1, from https://odr.chalmers.se/bitstream/20.500.12380/300650/1/E2019_128.pdf
  » https://odr.chalmers.se/bitstream/20.500.12380/300650/1/E2019_128.pdf
• Skrzypkowski, K., Korzeniowski, W., Zagórski, K., & Zagórska, A. (2020). Adjustment of the yielding system of mechanical rock bolts for room and pillar mining method in stratified rock mass. Energies, 13(8), 2082. http://dx.doi.org/10.3390/en13082082
  » http://dx.doi.org/10.3390/en13082082
• Soligo, V. (2012). Indicadores: conceito e complexidade do mensurar em estudos de fenômenos sociais. Estudos em Avaliação Educacional, 23(52), 12-25. http://dx.doi.org/10.18222/eae235220121926
  » http://dx.doi.org/10.18222/eae235220121926
• Sousa, J. C. C., Fernandes, R. S., Luz, R. M. N., Santos, I. R., Silva, B. L. F., & Santos Neto, I. (2022). Description of the IPCC mining process and analysis of the profile of productivity losses applied by a mining company in northern Brazil. Research Square, 1-25. https://doi.org/10.21203/rs.3.rs-1308199/v1
  » https://doi.org/10.21203/rs.3.rs-1308199/v1
• Srivastava, A. K., & Sushil. (2015). Modeling organizational and information systems for effective strategy execution. Journal of Enterprise Information Management, 28(4), 556-578. http://dx.doi.org/10.1108/JEIM-09-2013-0071
  » http://dx.doi.org/10.1108/JEIM-09-2013-0071
• Srivastava, A. K., & Sushil. (2017). Alignment: the foundation of effective strategy execution. International Journal of Productivity and Performance Management, 66(8), 1043-1063. http://dx.doi.org/10.1108/IJPPM-11-2015-0172
  » http://dx.doi.org/10.1108/IJPPM-11-2015-0172
• Wårell, L. (2018). An analysis of iron ore prices during the latest commodity boom. Mineral Economics, 31(1), 203-216. http://dx.doi.org/10.1007/s13563-018-0150-2
  » http://dx.doi.org/10.1007/s13563-018-0150-2
• Xu, L., Peng, X., Pavur, R., & Prybutok, V. (2020). Quality management theory development via meta-analysis. International Journal of Production Economics, 229, 107759. http://dx.doi.org/10.1016/j.ijpe.2020.107759
  » http://dx.doi.org/10.1016/j.ijpe.2020.107759
• Zanon, C., Hutz, C. S., Yoo, H. H., & Hambleton, R. K. (2016). An application of item response theory to psychological test development. Psicologia: Reflexão e Crítica, 29(1), 18. http://dx.doi.org/10.1186/s41155-016-0040-x
  » http://dx.doi.org/10.1186/s41155-016-0040-x

Publication Dates

• Publication in this collection
  30 Jan 2023
• Date of issue
  2023

History

• Received
  01 June 2022
• Accepted
  14 Oct 2022