EVALUATION OF HIGHER EDUCATION: AN EXPLORATORY STUDY ON THE COMPLIANCE OF SELF-EVALUATION REPORTS WITH TECHNICAL NOTE 065/2014/INEP/DAES/CONAES ¹

ABSTRACT: Institutional evaluation in Higher Education Institutions (HEIs) contributes to the improvement of management policies and to the qualification of academic management, encompassing teaching, research, and extension, as a way to consolidate their presence in society. Among the practices evaluated, the preparation and dissemination of final self-assessment reports stand out. The objective of this research is to identify the degree of compliance of the comprehensive self-assessment reports of selected HEIs with Technical Note No. 065/2014/INEP/DAES/CONAES. It is an exploratory study, based on the documentary analysis of the published reports. Through the application of a checklist, the adherence of the documents to the normative guidelines was verified. The results indicated a high degree of compliance (above 87.5%), evidencing advances in the systematization of the evaluation processes, but also pointing out gaps regarding financial sustainability and methodological detailing. The conclusions highlight the ongoing need for training the members of the Internal Evaluation Committees (CPAs) and suggest the adoption of artificial intelligence tools to strengthen and qualify the self-assessment reports, broadening their role as strategic instruments for management and institutional self-regulation.

Keywords:
institutional self-assessment; higher education; Sinaes; compliance; technical note

RESUMO:

A avaliação institucional de Instituições de Educação Superior (IES) contribui para o aprimoramento das políticas de gestão e para a qualificação da gestão acadêmica, abrangendo ensino, pesquisa e extensão, como maneira de consolidar sua presença na sociedade. Entre as práticas avaliadas, destaca-se a elaboração e publicização dos relatórios finais de autoavaliação. Assim sendo, o objetivo desta pesquisa é identificar o grau de conformidade dos relatórios de autoavaliação integral de IES selecionadas em relação à Nota Técnica 065/2014/INEP/DAES/CONAES. Trata-se de um estudo de caráter exploratório, baseado na análise documental de relatórios publicados. Por meio da aplicação de um checklist, procedeu-se à verificação da aderência dos documentos às orientações normativas. Os resultados indicaram elevado grau de conformidade (acima de 87,5%), evidenciando avanços na sistematização dos processos avaliativos, mas também, apontaram lacunas quanto à sustentabilidade financeira e ao detalhamento metodológico. As conclusões indicam a necessidade constante de capacitação dos membros que formam as CPAs e a sugestão de uso de ferramentas de inteligência artificial para fortalecer e qualificar os relatórios de autoavaliação, ampliando sua função como instrumento estratégico de gestão e de autorregulação institucional.

Palavras-chave:
autoavaliação institucional; educação superior; sinaes; conformidade; nota técnica

RESUMEN:

La evaluación institucional en las Instituciones de Educación Superior (IES) contribuye al perfeccionamiento de las políticas de gestión y a la cualificación de la gestión académica, abarcando la enseñanza, la investigación y la extensión, como forma de consolidar su presencia en la sociedad. Entre las prácticas evaluadas, se destaca la elaboración y divulgación de los informes finales de autoevaluación. El objetivo de esta investigación es identificar el grado de conformidad de los informes de autoevaluación integral de IES seleccionadas en relación con la Nota Técnica Nº 065/2014/INEP/DAES/CONAES. Se trata de una investigación de carácter exploratorio, basada en el análisis documental de los informes publicados. A través de la aplicación de una lista de verificación, se comprobó la adherencia de los documentos a las directrices normativas. Los resultados indicaron un elevado grado de conformidad (superior al 87,5%), evidenciando avances en la sistematización de los procesos evaluativos, pero también señalando lagunas en cuanto a la sostenibilidad financiera y al detalle metodológico. Las conclusiones señalan la necesidad permanente de formación de los integrantes de las Comisiones Propias de Evaluación (CPAs) y sugieren el uso de herramientas de inteligencia artificial para fortalecer y cualificar los informes de autoevaluación, ampliando su función como instrumento estratégico de gestión y autorregulación institucional.

Palabras clave:
autoevaluación institucional; educación superior; SINAES; conformidad; nota técnica

INTRODUCTION

Institutional evaluation is an essential element of the higher education ecosystem, guided by regulations and principles. Following the enactment of Law number 10,861 of April 14, 2004, which established the National System for the Evaluation of Higher Education (SINAES-Sistema Nacional de Avaliação da Educação Superior), a legal framework was created to improve evaluation, particularly with the establishment of Self-Assessment Committees (CPAs for its name in Portuguese) within Higher Education Institutions (HEIs). The Law led to the implementation of institutional self-assessment reports, which began to consider different dimensions. This article aims to verifying the conformity of comprehensive institutional self-assessment reports with the requirements of Technical Note 065/2014/INEP/DAES/CONAES, hereinafter referred to as NT65, from different HEIs.

Starting from the objective of identifying the degree of conformity of the comprehensive self-assessment reports of selected HEIs in relation to NT65, this study is characterized as exploratory research, as it intends to articulate theory and practice in the context of the routines of the CPAs (Institutional Self-Assessment Committees). By deepening the knowledge about the conformity of institutional self-assessment reports with NT65, the study aims to understand a field with little bibliographic production. According to Lanari (2024), “exploratory projects are indicated in situations such as the absence of previous studies capable of providing precise answers” (p. 190). Therefore, exploratory research is not only based on previously formulated hypotheses, but also constitutes a movement of articulation with practice, in order to highlight existing gaps.

In each HEI (Higher Education Institution), the CPAs are responsible for organizing and compiling the institutional evaluation data. To support this procedure, NT65 (Technical Note 65) offers guidelines and suggestions for adjustments in the preparation of reports, allowing for conciseness and better distribution of the elements to be presented, without prejudice to the different administrative categories. Given the bibliographic gap regarding procedures for preparing reports, the proposed empirical investigation of already prepared documents allows us to establish the problem of this study, in the following direction: What is the degree of conformity of the comprehensive self-assessment reports of the selected HEIs in relation to Technical Note No. 65/2014?

Beginning with the identified problem, this study outlines the theoretical framework, providing an overview of the context surrounding CPAs and NT65. Following that, the methodological approach is detailed, including the criteria for selecting higher education institutions (HEIs) and the development of the analysis instrument. Finally, the results obtained are presented, accompanied by a critical discussion.

This exploratory study investigated the reports produced by the following higher education institutions (HEIs): Faculdade Vicentina (FAVI), a confessional and private non-profit institution based in Curitiba (PR); Universidade Federal do Rio Grande (FURG), a public federal institution located in Rio Grande (RS); and Universidade do Extremo Sul Catarinense (UNESC), a community-based and also private non-profit institution located in Criciúma (SC). All of them make their self-assessment reports available on their respective websites². The HEIs selected for this study present different administrative categories, allowing the instrument to be validated in different contexts (Lozada, 2019). For this reason, an exploratory study is one in which the “researcher selects the elements they have access, admitting that these may, in some way, represent the universe” (Prodanov; Freitas, 2012, p. 98).

This research is validated by the introduction of innovative perspectives that represent opportunities for developing methodologies to assist CPAs and other management bodies of Higher Education Institutions (HEIs) in validating reports and producing documents aimed at generating quality indicators. It is proposed that this methodology be adapted across various administrative categories to ensure the effective execution of evaluation activities. An additional motivation for this study arises from the fact that HEIs, after submitting their reports to the e-MEC system, do not receive feedback concerning their compliance with the NT65 structure and its associated indicators. Consequently, the findings of this study offer a valuable tool to support institutions, particularly CPA members who are new to institutional evaluation, in identifying discrepancies in the reports prepared in accordance with NT65.

The conclusions from this exploratory study underscore its contributions to relatively underexplored and theorized topics within the realm of higher education evaluation. They highlight the potential for further research in this area. Specifically, the opportunity to value the technical productions carried out within higher education institutions stands out, so that self-assessment reports transcend the function formalities instruments and become effectively incorporated into the strategic planning and bolster institutional development.

THEORETICAL FOUNDATION

The term self-assessment was formally included in the context of Brazilian higher education starting in 1993, with the Institutional Evaluation Program of Brazilian Universities (PAIUB-Programa de Avaliação Institucional das Universidades Brasileiras) of the Ministry of Education (MEC), created it “as an initial stage of a process that, once triggered, extended to the entire institution and was completed with external evaluation” (Inep, 2009, p. 27). After other experiences with higher education evaluation, such as the National Course Examination (ENC-Exame Nacional de Cursos), Law number 10,861/2004 was instituted in 2004 (Brazil, 2004), expanding the vision adopted by PAIUB, which gave rise to the National System for the Evaluation of Higher Education (SINAES).

Magalhães and Rodrigues (2020) emphasize that, despite criticisms of the models and the possibility of their continuous improvement, evaluation is recognized as an important field “of reflection that treats evaluation as the construction of knowledge, that is, as a scientific field” (p. 914). As a system, SINAES is structured by three main components: institutional evaluation, course evaluation, and student performance evaluation, as illustrated in Graph 1.

Graph 1
- Global Synthesis of the Main Components of SINAES

The synthesis illustrated in Graph 1 provides a didactic visualization of elements that initially appear distinct and unrelated. However, as components of a system, they are, in fact, interrelated. The findings from on-site evaluations of higher education institutions and their courses, alongside the National Student Performance Examination (ENADE), should be integrated into the institutional self-evaluation process. This integration is vital as it has a direct influence on institutional planning and governance, thereby affecting institutional evaluation outcomes. The National Institute of Educational Studies and Research Anísio Teixeira (INEP) asserts that "self-evaluation fulfills its function when the Institutional Self-Assessment Committee (CPA) identifies the institution's strengths and weaknesses and suggests modifications based on the analyses conducted" (Brazil, 2011, p. 89).

With the publication of Technical Note number 65/2014, hereinafter referred to as NT65, it becomes possible to understand institutional self-assessment in a comprehensive manner, indicating an alignment of the components that form the whole of the evaluation. In practice, this alignment was already evident, since the institutional self-assessment report is one of the documents that support on-site evaluations (Oliveira; Rothen, 2024). Considering that each component of SINAES allows for extensive studies, this article focuses on internal evaluation, also called self-assessment, and its final educational product: the institutional self-assessment report.

Morosini (2021) corroborates this view by pointing out the importance of institutional self-evaluation as a “continuous process of knowledge construction,” which allows one to “understand the meanings of the set of activities developed to improve educational quality, achieve greater social relevance, identify weaknesses, strengths and potentials, and establish strategies to overcome problems” (Morosini, 2021, p. 219). Thus, self-evaluation cannot be treated as an isolated, fragmented event, as this would make it difficult to obtain a broad, systematic view of institutional functioning.

Self-assessment primarily enables the institution to understand itself, allowing it to identify the gap between its current reality and the objectives and goals established in the Institutional Development Plan (PDI-Plano de Desenvolvimento Institucional). The strategic role of the PDI in educational institutions is a relevant aspect to be highlighted. Sousa and Griboski (2024) state that the PDI “has become an important benchmark of institutional quality in external and internal evaluation processes” (Sousa; Griboski, 2024, p. 4), since it establishes the guiding principles for planning in higher education institutions. Brito et al. (2021) affirm that institutional planning is strategic for the management of higher education institutions, enabling to monitoring of planned actions and their adjustments when necessary.

Among the institutional policies that comprise the PDI, it is fundamental to foresee guidelines for institutional evaluation processes, ensuring alignment with current legislation and the institutional self-evaluation project. In general terms, this institutional evaluation project establishes how the self-evaluation will be conducted throughout the established period, resulting in the production of partial self-evaluation reports (in the first and second years) and full reports (in the third year). To standardize the institutional self-evaluation reports, ensuring their availability to the MEC and the INEP as input for evaluation and regulatory processes, NT65 was created.

Technical Note number 065/2014/Inep/Daes/Conaes - NT65

The genesis of NT65 was created from debates at the National CPA Seminars, promoted by the Directorate of Higher Education Evaluation of INEP (DAES/INEP). These discussions resulted in the joint organization and approval of NT65 by DAES, INEP, and the National Commission for Higher Education Evaluation (CONAES-Comissão Nacional de Avaliação da Educação Superior), as recorded in the Minutes of the 112th CONAES Meeting of December 9, 2014, especially in its Annex 2 (Brazil, 2014). Furthermore, the document clarifies the group's understanding of self-evaluation and its role:

Internal evaluation, also called self-evaluation, in accordance with the Institutional Development Plan (PDI) of the HEI, should be seen as a process of self-knowledge conducted by the Self-Assessment Committee (CPA), involving the actors who work in the institution, to analyze the academic activities developed (Brazil, 2014, p. 2).

NT65 aims to improve the quality of higher education. This regulation details the criteria and essential information for the composition of institutional self-assessment reports, ensuring that the data presented are clear, coherent, and organized in a standardized way, but without hindering the unique characteristics of each higher education institution. Furthermore, NT65 highlights the importance of institutional transparency, a fundamental aspect for consolidating social control and the public accountability of the commission in the evaluation process.

Transparency is a central principle in the NT65 guidelines. According to Griboski, Peixoto, and Hora (2018), well-organized, clear, and easily accessible reports allow both the academic community and society to monitor and understand the actions developed by institutions, as well as their results. This type of practice strengthens social responsibility and contributes to the credibility and legitimacy of higher education institutions in the eyes of society.

According to Cavalcanti (2024, p. 249), institutional self-assessment processes “use qualitative methodologies that, prioritizing debate, reflection, and the narratives of the actors involved, assume a formative character, conferring legitimacy to the process.” Therefore, an institutional self-assessment that promotes the broad participation of different segments of the academic community and civil society, as well as the publication of results, contributes to the evolution of university management and strengthens the institutional mission.

As a normative document, NT65 results in an educational product in the form of a report, with a specific and ordered structure. The self-assessment report must cover five axes: (I) Institutional Planning and Evaluation; (II) Institutional Development; (III) Academic Policies; (IV) Management Policies; and (V) Infrastructure. These axes encompass 10 dimensions foreseen by SINAES: Mission and Institutional Development Plan; Social Responsibility of the Institution; Policies for Teaching, Research and Extension; Communication with Society; Student Support Policy; Personnel Policies; Organization and Management of the Institution; Physical Infrastructure; Planning and Evaluation; and Financial Sustainability (Inep, 2014).

The significance of NT65 lies in its establishment of a normative framework that guides the preparation of self-assessment reports, capturing the institutional reality while maintaining a focus on the quality indicators set forth by the MEC (Ministry of Education). In this context, Cavalcanti (2024) makes a notable contribution by emphasizing the evolution of institutional evaluation processes, which are crucial for enhancing the analyses derived from these evaluations. According to the author, the responsibility for these processes “belongs to the various actors within the university community (students, faculty, and technical-administrative staff)” (Cavalcanti, p. 249). Additionally, alumni and representatives of organized civil society play a vital role by offering their perspectives on Higher Education Institutions (HEIs) and their programs during the self-assessment process. These factors underscore the importance of the self-assessment report as an essential tool for effective university management.

Although NT65 establishes clear criteria for self-assessment reports, international literature points to the need to broaden the assessment to more comprehensive dimensions, encompassing social and environmental impacts. In recent years, HEIs have been called upon to consider not only compliance with internal or national regulations, but also their effective contribution to broader social, environmental, and economic impacts. According to the UNESCO Higher Education Global Data Report, there is a growing international demand for data showing how HEIs influence social, equity, and sustainable development indicators beyond their institutional walls (UNESCO, 2022). This trend is also materialized in the use of impact rankings, such as the Times Higher Education Impact Rankings, which measures how universities respond to the Sustainable Development Goals and evaluate institutional practices.

CRITICAL DISCUSSION

Institutional self-assessment, conducted by the CPAs is an important process for HEIs to maintain continuous improvement. As highlighted by Brito et al. (2021), “the CPA can create a permanent space for dialogue between the HEI's management and the academic community to support the management in the elaboration of its planning” (pp. 76-77). This dialogical and participatory dynamic is vital to ensure that evaluations are not merely an application of bureaucratic norms to meet regulatory processes, but rather an effective instrument for management and improvement of educational quality.

The implementation of NT65 faces unique challenges due to the diverse institutional contexts present in Brazil. Reports generated from institutional self-assessment processes serve as vital inputs for strategic planning and the ongoing development of the institution. Within each higher education institution (HEI), teaching, research, extension, and management policies are articulated in their respective PDIs. According to Phaiffer (2020), the PDI “should be created and considered an effective guiding plan for the management plan” (p. 222). As a strategic document, the PDI establishes operational guidelines for the institution, covering everything from course creation to resource allocation and the formulation of institutional policies.

According to the policies outlined in the PDIs, the activities conducted by the HEIs are evaluated in each cycle, producing data that is subsequently analyzed and documented in the self-assessment reports prepared annually by the CPAs in line with the NT65 guidelines. This study opted for an analysis of the report characterized as comprehensive (INEP, 2014, p. 4), as it revisits the two partial reports and integrates the insights from the third year of the cycle.

This exploratory study is characterized as applied research with a qualitative approach and exploratory nature (Prodanov; Freitas, 2012, p. 98), aiming to understand the degree of conformity between the comprehensive self-assessment reports of the selected HEIs in relation to NT65. The data collection (survey) was carried out using a checklist developed by the authors of this study to identify, together with the selected reports, the degree of conformity with NT65, which was a reference for the qualitative analysis. In this checklist, the authors defined evaluation items and response options organized in the form of a three-level scale, as shown in Table 1.

Table 1
Demonstration of the scale used for the checklist

Scale	Description
1 - Non-Compliant	The report does not meet, or barely meets, the requirements of NT65, presenting significant inconsistencies and lacking essential elements.
2 - Partially Compliant	The report partially meets NT65, addressing some criteria, but has gaps or inconsistencies that compromise its adequacy.
3 - Fully Compliant	The report is in full compliance with NT65, fulfilling all requirements in a clear, coherent, and complete manner.

• Source: Prepared by the authors (2025).
• The three-level scale was adopted as a way to test the developed indicators, aiming for a common analysis of the reports from higher education institutions that fall into three different administrative categories.

Contextualization of the selected HEIs

All HEIs linked to the federal education system, across different administrative categories, must have an established CPA, responsible for organizing and “coordinating the internal evaluation processes of the HEI; systematizing and providing information requested by the National Institute of Educational Studies and Research Anísio Teixeira (INEP)” (Morosini, 2021, p. 442). Therefore, the HEIs selected for this study make their respective CPA compositions available on their websites.

Faculdade Vicentina (FAVI): the Vincentian charism is characterized by the education of new generations, originating in 1625, dedicated to the formation of children and young people in the interior of France (Vicentina Faculty, 2023, p. 7). Faculdade Vicentina inherited the commitments to the integral human formation of the Vincentian charism and is located in the city of Curitiba (PR). It is a private, confessional college, belonging to the federal education system and dedicated to teaching, extension, and scientific initiation. Currently, it offers 2 undergraduate courses and postgraduate lato sensu courses in the area of ​​human sciences. Its mission is “To act in higher education with a view to the integral promotion of the human being, guaranteeing the quality of teaching to form citizens committed to the ethical, social, cultural, and political development of society and to the care of the planet” (Faculdade Vicentina, 2023, p. 10). FAVI has a well-established and active CPA to improve institutional self-assessment processes efficiently.

Universidade Federal do Rio Grande (FURG): The Universidade Federal do Rio Grande is a public higher education institution located in the municipality of Rio Grande, in the far south of the state of Rio Grande do Sul. Founded in 1969, the university emerged from the merger of the city's first four higher education schools. Currently, FURG offers 70 undergraduate courses and more than 70 postgraduate courses, with an academic community encompassing more than 9,000 students, approximately 1,050 administrative and educational technicians, and approximately 860 faculty members. Its structure is multi-campus, with units located in municipalities in the region, such as São Lourenço do Sul, Santo Antônio da Patrulha, and Santa Vitória do Palmar (FURG, 2019). FURG's mission is “To promote the advancement of knowledge and full education with excellence, training professionals capable of contributing to human development and the improvement of socio-environmental quality” (2024, p. 10). Its CPA was created in 2004, after the enactment of the SINAES Law, and implemented the following year. In 2016, its regulations were updated to adapt to the new reality of the institution, considering the changes in the statute and the creation of new campuses (Furg, 2016).

Universidade do Extremo Sul Catarinense (UNESC): The Universidade do Extremo Sul Catarinense was originated in 1968, when the Fundação Educacional de Criciúma (FUCRI), its maintaining entity, was established by then-mayor Rui Hülse through a Municipal Law. Between 1970 and 1975, the first higher education schools began offering courses in education and sciences, physical education, technology, accounting, and administration. With the unification of the schools, the Union of Colleges of Criciúma (UNIFACRI-União das Faculdades de Criciúma) emerged in 1991, which in 1997 became UNESC, officially recognized as a university. UNESC has its main campus in Criciúma, Santa Catarina, and is classified as a community-based, private, non-profit institution. In 2023, the institution had 63 undergraduate courses (both in-person and distance learning), 8 master's programs, and 5 doctoral programs, 306 research projects, 194 extension projects, 13,607 students, considering all levels of education, 671 faculty members, and 708 administrative staff (Unesc, 2023, 2024). Its mission is “To educate, through teaching, research, and extension, to promote the quality and sustainability of the living environment” (Unesc, 2024, p. 22). Furthermore, the institution's PDI has its strategic guidelines based on self-assessments carried out by its CPA in partnership with the Institutional Evaluation Sector (SEAI) (Unesc, 2024).

The three HEIs have distinct administrative categories: private confessional college, private non-profit university classified as community-based, and public university. Furthermore, their headquarters are located in Curitiba (PR), Criciúma (SC), and Rio Grande (RS), respectively, demonstrating the geographical scope of this study, considering the states of the Southern region of Brazil.

Application of the checklist for analyzing HEIs' reports.

The compliance analysis of the selected higher education institutions' reports with NT65 was conducted through the development of a checklist featuring evaluation items and response options organized on a three-level scale. Upon its organization, the checklist was applied with the assistance of Generative Artificial Intelligence (GAI), which represents an innovative element in the evaluation processes of higher education. This approach is predicated on the belief that combining technological tools with critical human review can enhance the comprehensiveness and effectiveness of AI-powered analyses.

The use of AI tools in academic research is expanding considerably. According to Kaufman (2022), AI is being widely used to optimize the interpretation of information. Despite this, AI “is only a statistical probability model based on data, ‘light-years’ away from the complexity of the human brain” (Kaufman, 2022, p. 32), which highlights the importance of humans in validating the results obtained. Along the same lines, Machado et al. (2024) emphasize that, although the application of AI in academic research enhances the analysis and systematization of textual content, it is fundamental that the researcher actively participates in the process to guarantee the reliability and accuracy of the information generated.

To prevent interference from machine training during the checklist analyses conducted with AI, the tests were carried out by a single team member. Initially, only ChatGPT was utilized, as DeepSeek did not facilitate the complete upload of reports during the machine training phase. Following the initial analysis supported by IAGen, the authors of this study conducted a comprehensive review of the gathered information to ensure its accuracy.

The following section outlines the evaluated items and the degree of conformity assessed by IAGen, as validated by the authors of this study. Through this evaluation process, we can confidently assert the high level of reliability associated with IAGen for the exploratory study of report conformity. The conformity of the reports, as assessed by IAGen, is categorized according to the criteria established in Table 1: (1) Non-Compliant, indicating that the report does not meet, or marginally meets, the requirements of NT65, thereby revealing significant inconsistencies and/or the absence of essential elements; (2) Partially Compliant, denoting that the report meets some aspects of NT65 but contains gaps or inconsistencies that undermine its overall adequacy; and (3) Fully Compliant, signifying that the report adheres entirely to NT65, fulfilling all requirements in a clear, coherent, and comprehensive manner.

Results obtained from the checklist applied to HEIs.

Based on the information presented in the previous section, the results obtained from applying the checklist with the support of the IAGen tool will be presented here. It is important to highlight that the applications, along with the self-assessment reports, were carried out separately for each HEI, respecting the arrangement of the indicators shown in Table 2.

Table 2
Demonstration of the application of the checklist to the HEI reports

Indicator	RESULTS
	FAVI	FURG	UNESC
INTRODUCTION SECTION
The report shows data about the institution and its self-assessment strategic planning.	TC	TC	TC
The introduction contains the composition of the CPA, the year to which the report refers, as well as whether the report is partial or complete.	TC	TC	TC
METHODOLOGY SECTION
The report details the methodology used and describes the instruments used to collect the data.	TC	TC	TC
The methodology describes the segments of the academic community and civil society consulted for the self-assessment process.	TC	TC	TC
There is a record of the techniques used for data analysis.	PC	TC	TC
DEVELOPMENT SECTION
Axis 1. The axis is identified in reference to the composition cycle of the self-assessment report.	TC	TC	TC
Axis 1. There is evidence of the planning and evaluation dimension in the self-assessment report.	TC	TC	TC
Axis 2. The axis is identified in reference to the composition cycle of the self-assessment report.	TC	TC	TC
Axis 2. There is evidence of the Mission and Institutional Development Plan dimension in the self-assessment report.	TC	TC	TC
Axis 2. There is evidence of the Institution's Social Responsibility dimension in the self-assessment report.	TC	TC	TC
Axis 3. The axis is identified in reference to the composition cycle of the self-assessment report.	TC	TC	TC
Axis 3. There is evidence of the dimension of Policies for Teaching, Research and Extension in the self-assessment report.	TC	TC	TC
Axis 3. There is evidence of the Communication with Society dimension in the self-assessment report.	TC	TC	TC
Axis 3. There is evidence of the Student Support Policy dimension in the self-assessment report.	TC	TC	TC
Axis 4. The axis is identified in reference to the composition cycle of the self-assessment report.	TC	TC	TC
Axis 4. There is evidence of the Personnel Policies dimension in the self-assessment report.	TC	TC	TC
Axis 4. There is evidence of the Organization and Management dimension of the Institution in the self-assessment report.	TC	TC	TC
Axis 4. There is evidence of the Financial Sustainability dimension in the self-assessment report.	PC	TC	TC
Axis 5. The axis is identified in reference to the composition cycle of the self-assessment report.	TC	TC	TC
Axis 5. There is evidence of the Physical Infrastructure dimension in the self-assessment report.	TC	TC	TC
DATA ANALYSIS SECTION
There is evidence that the information in the report was analyzed and appropriated by segments of the internal community.	TC	TC	TC
The report presents a diagnosis of the HEIs, highlighting progress and challenges faced.	TC	TC	TC
The report demonstrates the achievements planned in the PDI, justifying any failure to reach certain objectives and goals.	PC	TC	TC
SECTION: ACTIONS BASED ON ANALYSIS
The report outlines action plans or improvement strategies based on the results presented and analyzed.	TC	TC	TC

• Legend: TC = Fully compliant; PC = Partially compliant Source: Prepared by the authors (2025).

The results obtained from the analysis of Table 2 are from the 24 evaluation items listed in the left-hand column. The results were FAVI 87.5% (TC) and 12.5% ​​(PC); UNESC 100% (TC); FURG 100% (TC). A brief analysis by section also highlights that:

In the Introduction section, all three reports are fully compliant with the necessary items outlined in NT65: data from the HEI and the self-assessment plan, the composition of the CPA, the year to which the report refers, and whether it is partial or complete. Compliance was 100% for all three HEIs. This data is relevant because it is standard practice to compile a public report with contextualization of the HEI and certain information related to the document, based on the principle that it can be accessed by any person or institution, whether or not they are familiar with the SINAES guidelines, but who are interested in obtaining information related to the report. From this perspective, as NT65 guides in its section 3.1, it is evident which elements should be addressed in the introduction of the HEI's self-assessment report, facilitating the composition of this section by the CPAs.

When examining the Methodology Section, it indicates the necessary elements for composing the HEI's report: methodology and application instruments, participating segments, and techniques used for data analysis. The results revealed a lack of detail regarding the analysis techniques employed by one HEI, which is a point of concern. This result aligns with the final report of the meeting promoted by INEP with the CPAs, incorporating the suggestions presented (Brazil, 2010). In that event, one of the final suggestions highlighted the need for “INEP to conduct workshops for the training of CPA coordinators” (p. 99). Considering that CPA members do not always possess knowledge involving research or report writing, and that current legislation, as well as CONAES, does not establish a specific academic profile beyond the representativeness of institutional segments, these members must have the opportunity for training related to scientific methodology so that they can contribute to making the report not only adequate in terms of NT65, but also ensuring that the results and analyses are not subject to doubt. On the other hand, depending on the structure and complexity of a higher education institution, the self-assessment report becomes more robust, and it is possible that some elements may unintentionally be omitted when writing the methodological strategies.

When reaching the Development Section, according to NT65, the 5 axes and their dimensions are addressed, observing the following:

Axis 1: NT65 requires identification of the Institutional Planning and Evaluation axis and its respective dimension, Planning and Evaluation, and whether there is evidence of this axis and dimension in the report structure. In this axis, compliance was 100% for the three HEIs.

Axis 2: In this axis, NT65 requires the identification of Institutional Development and two dimensions: Mission and Development Plan, and Social Responsibility. In this second axis, 100% compliance was also identified for the three HEIs.

Axis 3: NT65 requires the identification of the Academic Policies axis and the existence of three dimensions in the report structure: policies for teaching, research and extension, communication with society, and student support policy. In this axis as well, compliance was 100% for the three HEIs.

Axis 4: NT65 requires the identification of the Personnel Policies axis and the existence of three dimensions in the report structure: Personnel Policy, Organization and Management of the Institution, and Financial Sustainability. The application of the checklist identified the absence of a detailed description of planning for the financial sustainability of an HEI. In the final report of the meeting promoted by INEP with the CPAs (Brazil, 2010), it was identified at the time that the "evaluation of the financial sustainability of the HEIs presented in the reports did not allow for the achievement of knowledge of the reality" (p. 77). With the data they had, the author inferred that the dimension might not have been evaluated. Another possibility that the authors of this study assume involves the fact that the self-assessment report is a public document, presenting important but also sensitive information about higher education institutions, such as financial sustainability.

Axis 5: NT65 requires identification of the Physical Infrastructure axis and whether there is evidence of the dimension of the same name in the report structure. In this axis, compliance was 100% for the three HEIs.

Data Analysis Section: NT65 addresses the importance of data appropriation by HEI segments, highlights advances and challenges, and the identification of achievements based on the self-assessment objectives. It is noticeable that one HEI does not include a detailed description of achievements and their relationship with the objectives of the higher education assessment process. Brito et al. (2021) highlight that "planning for any institution is a key piece of management [...]" (p. 9). Self-assessment reports can provide important input for institutional planning, "and should have an educational character and be geared towards decision-making by the HEI, collective participation, institutional improvement and self-regulation" (Sousa; Griboski, 2024, p. 5). This is the goal that CPAs tend to have, so that the report does not become a simple bureaucratic and regulatory action, posted on e-MEC and accessible to the commissions designated for the renewal of institutional and course regulatory acts. Brazil (2010) corroborates this perception when it highlights that the CPAs did not prepare a simple activity report, taking care “to provide subsidies resulting from the process implemented by them” (p. 88).

Section Actions based on the analysis: the results of this section indicate 100% compliance of the reports with the necessary elements outlined in NT65.

Although the analyzed reports showed high levels of regulatory compliance, it is important to note that the international literature points to limitations in this type of approach. International assessment tools such as the Sustainability Assessment Tools (SAT) demonstrate that, although many instruments focus on internal institutional performance, there is a gap in measuring impacts that go beyond the organizational scope. According to Findler et al. (2019), when analyzing 19 SAT models and 1,134 different indicators, it was found that most metrics are performance or “proxy” indicators, with few indicators of direct or indirect impact outside the institution, especially with regard to social and environmental dimensions. This contrast highlights that, in the self-assessment reports analyzed, although there is regulatory compliance, elements for measuring impact, such as long-term effects on the community, institutional culture, or the environment, remain poorly considered, suggesting fertile ground for evaluative enrichment.

INNOVATIVE PERSPECTIVES

The research is justified by its novel theme, considering that the authors of this study carried out a bibliographic survey in the Scielo, Scopus, and Web of Science (WoS) databases, applying a search string (Table 3) developed from the research question.

Table 3
Search string for bibliographic survey in databases

Search String

("autoavaliação institucional" OR "relatórios de autoavaliação" OR "institutional self-assessment" OR "self-assessment reports") AND ("Nota Técnica 065" OR "NT 065/2014" OR "Technical Note 065" OR "SINAES") AND ("avaliação institucional" OR "gestão universitária" OR "institutional evaluation" OR "university management") AND ("ensino superior" OR "qualidade na educação superior" OR "higher education" OR "quality in higher education") AND ("regulação" OR "conformidade regulatória" OR regulation" OR "regulatory compliance) AND ("avaliação educacional" OR "processo avaliativo" OR "educational assessment" OR "evaluation process") AND ("critérios de avaliação institucional" OR "gestão e regulação do ensino superior" OR "institutional evaluation criteria" OR "higher education governance")

• Source: Prepared by the authors (2025).

The result of this search did not present academic productions related to the theme of this study, a fact that motivated the continuation and deepening of the research. The proposal brings a new perspective to the evaluation of higher education through the reports produced by the CPAs, based on the regulation of NT65.

One of the innovative elements of this research was the construction and validation of a systematized checklist for analyzing the conformity of institutional self-assessment reports. The proposed instrument, composed of 24 indicators organized into axes and dimensions, allows its replicability in other investigations, strengthening the standardization of institutional evaluation criteria as advocated by NT65.

The possibility of replicating the checklist in different types of higher education institutions, such as universities, university centers, and independent colleges, expands its innovative potential. The structure by axes and dimensions standardizes the analysis criteria but maintains openness to contextual adaptations, which favors both comparability and personalization of institutional diagnoses. Thus, the checklist not only systematizes but also democratizes the evaluation process, allowing institutions with different profiles and degrees of complexity to adopt a more effective self-evaluation culture aligned with the legal framework of SINAES.

Another innovative element of this research is the application of Generative Artificial Intelligence (GAI) as a support tool to analyze self-assessment reports. This approach represents a methodological innovation that can transform the way institutions conduct their internal analyses, but beyond that, it stimulates and fosters new methods of external evaluation. Kaufman (2020) highlights that, although GAI still depends on human validation, its application can expand analytical capacity, especially in repetitive and standardized activities. The most recent literature emphasizes that the integration of emerging technologies and the critical and reflective action of researchers enhances the reliability of evaluation processes (Machado et al., 2024).

In this exploratory study, the checklist was applied to verify the conformity of the self-assessment reports based on the NT65 guidelines. IAGen was used as an auxiliary instrument in the initial stage of the analysis, and the results were subsequently validated by the authors. According to Machado et al. (2024b), the hybrid application using IAGen and human review “for the search for themes that indicate innovation and gaps for academic research, enabling significant performance gains in scientific research through greater assertiveness and agility in the work” (p. 14), has been developing safely in the academic environment. The experience of using IAGen tools, combined with rigorous monitoring by researchers, corroborates the innovative and rapid contribution of mixed research processes and feedback to stakeholders.

The innovation of this research is also revealed in the diversity of the HEIs evaluated, which comprise three distinct administrative categories: a public university (FURG), a community university (UNESC), and a private confessional college (FAVI). This diversity, articulated with the geographical distribution in the three states of the Southern Region of the country (RS, SC, and PR), allows for broadening the possibilities of analysis regarding the application of NT65 in different institutional contexts, respecting their organizational, cultural, and structural realities. As highlighted by Cavalcanti (2024), institutional evaluation needs to consider the specificities of HEIs, as these constitute plural organizations that require technical and ethical-political capacities in their evaluation processes.

The choice to analyze the comprehensive reports from the third year of the evaluation cycle broadens the analytical potential of the research. Unlike traditional external evaluations—which operate with standardized instruments and specific cycles—the analysis of comprehensive self-evaluation reports enables capturing the institutional maturation process over three years. The strategy adopted in this research offers a new methodological possibility by proposing the use of comprehensive reports as a primary source for analyzing institutional quality, which reinforces the formative perspective of evaluation advocated by Cavalcanti (2024) and Morosini (2021).

The exploratory study also stands out as a potential innovation in the field of rankings. According to the proposal, it opens the way for the construction of an alternative criterion for institutional ranking, not based exclusively on quantitative performance indicators, but rather on the quality of the evaluation process conducted by the CPAs (Institutional Evaluation Committees). This proposal addresses a gap in current evaluation practices and, at the same time, values ​​self-evaluation as an instrument of institutional management and self-knowledge. As Magalhães and Rodrigues (2020) emphasize, evaluation should be understood as knowledge construction, and not just as verification of results. In this sense, the ranking suggested in this work considers the evaluative maturity of HEIs (Higher Education Institutions) as a quality indicator, based on the level of conformity of their reports with NT65 (Technical Standard 65).

FINAL CONSIDERATIONS

This exploratory study aimed to identify the degree of conformity of the comprehensive self-assessment reports of three higher education institutions (HEIs) from different administrative categories, FAVI, FURG, and UNESC, in relation to Technical Note 065/2014/INEP/DAES/CONAES. The results indicated that the HEIs have a degree of conformity in relation to the technical note and, positively, revealed results above 87.5% conformity. Of the twenty-four evaluation items, only one of the HEIs obtained three indicators evaluated as Partially Compliant, while all the others were Fully Compliant.

The methodological choice of using comprehensive reports as primary sources enabled a deeper understanding of the maturation of evaluation practices in higher education institutions (HEIs). In the cases analyzed, significant adherence to the criteria proposed by NT65 was identified, mainly in the areas related to institutional development, academic policies, and organizational structure. However, aspects such as the description of data analysis techniques, the approach to financial sustainability, and the monitoring of detailed goals in the PDI still require attention from the managers of one of the HEIs.

Furthermore, the combination of the checklist with IAGen tools, validated by human review, proved to be an effective and innovative strategy, providing methodological gains. The possibility of replicating the model applied in this study in different institutional contexts broadens the prospects for innovation for other HEIs and reinforces the role of self-assessment as a management and self-regulation tool, aiming at the qualification of these processes.

The research development process in dialogue with the available production reinforces the need for constant investment in training for CPA members, especially with regard to new information technologies, specifically artificial intelligence tools, considering the upcoming reports to be prepared. That is, once the proposal for developing evaluation items and the comparison process carried out by IAGen have been validated.

Based on the results found, we suggest future research to expand the analysis to the entire report preparation cycle in order to verify compliance with regulatory documents, so that it is possible to have a continuous and comprehensive understanding of all stages of the institutional self-assessment process. The self-assessment cycle also includes the partial reports of the first and second years (Inep, 2014, p. 4). These reports can provide valuable information on the evolution and adjustments made throughout the cycle. This approach broadens the scope of analysis and compliance verification, aiming to understand more broadly the trajectory and maturation of the evaluation practices conducted by the CPAs over time.

During the preparation of this study, we concluded that the institutional self-assessment process provides robust data for decision-making based on the diagnosis raised. Thus, a self-assessment process is, in the first instance, a formative step, as it favors the knowledge of institutional strengths and weaknesses. And, secondly, it can also be identified as a creative and proactive step for the team responsible for institutional management.

References

• BRASIL. Lei nº 10.861, de 14 de abril de 2004. Institui o Sistema Nacional de Avaliação da Educação Superior - SINAES e dá outras providências. Diário Oficial da União, n. 72, Seção 1, p. 1, 15 abr. 2004.
• BRASIL. MINISTÉRIO DA EDUCAÇÃO. Anexo 02 da Ata 112ª RO da Conaes MEC/INEP/DAES/CONAES, 9 dez. 2014. Disponível em: Disponível em: https://www.gov.br/participamaisbrasil/anexo-02-da-ata-112-ro-da-conaes Acesso em: 3 mar. 2025.
  » https://www.gov.br/participamaisbrasil/anexo-02-da-ata-112-ro-da-conaes
• BRITO, Renato de Oliveira; ZUIN, Antônia Carla de Oliveira; HATTGE, Larissa Nunes. Comissão Própria de Avaliação - CPA: sua atuação na construção do diálogo entre comunidade acadêmica e direção da IES. Avaliação: Revista da Avaliação da Educação Superior, Campinas, v. 26, n. 1, p. 68-88, abr. 2021.
• CAVALCANTI, Lourdes Maria Rodrigues. Autoavaliação institucional: significados e efeitos sobre as políticas educacionais na Universidade Federal da Paraíba Tese (Doutorado em Educação) - Universidade Federal da Paraíba, João Pessoa, 2024.
• FACULDADE VICENTINA. Plano de desenvolvimento institucional Curitiba: FAVI, 2023. Disponível em: Disponível em: https://drive.google.com/file/d/1vbVazLDsnunsM4Jt6zcPzro18aJm9_UM/view Acesso em: 14 fev. 2025.
  » https://drive.google.com/file/d/1vbVazLDsnunsM4Jt6zcPzro18aJm9_UM/view
• FINDLER, F.; SCHÖNHERR, N.; LOZANO, ; R. et al . Assessing the Impacts of Higher Education Institutions on Sustainable Development - An Analysis of Tools and Indicators Sustainability, v. 11, n. 1, p. 59, dez. 2018. DOI: 10.3390/su11010059.
  » https://doi.org/10.3390/su11010059
• INSTITUTO NACIONAL DE ESTUDOS E PESQUISAS EDUCACIONAIS ANÍSIO TEIXEIRA (INEP). Nota Técnica INEP/DAES/CONAES nº 065 Brasília: INEP, 9 out. 2014. Disponível em: Disponível em: https://download.inep.gov.br/educacao_superior/avaliacao_institucional/nota_tecnica/2014/nota_tecnica_n65_roteiro_relatorio_de_autoavaliacao_institucional.pdf Acesso em: 17 mar. 2025.
  » https://download.inep.gov.br/educacao_superior/avaliacao_institucional/nota_tecnica/2014/nota_tecnica_n65_roteiro_relatorio_de_autoavaliacao_institucional.pdf
• INSTITUTO NACIONAL DE ESTUDOS E PESQUISAS EDUCACIONAIS ANÍSIO TEIXEIRA (INEP). Sistema Nacional de Avaliação da Educação Superior (SINAES): análise dos relatórios de autoavaliação das Instituições de Educação Superior Brasília: INEP, v. 3, 2010.
• INSTITUTO NACIONAL DE ESTUDOS E PESQUISAS EDUCACIONAIS ANÍSIO TEIXEIRA (INEP). Sistema Nacional de Avaliação da Educação Superior (SINAES): da concepção à regulamentação 5. ed., rev. e ampl. Brasília: INEP , 2009.
• KAUFMAN, Dora. Desmistificando a inteligência artificial: reflexões éticas e sociais São Paulo: XYZ, 2022.
• LANARI, Alfredo. Um estudo exploratório preliminar sobre o lugar da comunicação na ciência. Multitemas, Campo Grande, v. 29, n. 72, p. 183-208, maio/ago. 2024. Disponível em: Disponível em: https://multitemas.ucdb.br/multitemas/article/view/4399/3019 Acesso em: 8 abr. 2025.
  » https://multitemas.ucdb.br/multitemas/article/view/4399/3019
• LOZADA, Gisele. Metodologia científica Porto Alegre: Sagah, 2019. 1 recurso online. ISBN 9788595029576. Acesso em: 9 abr. 2025.
• MACHADO, Ariél Philippi; HOFFMANN, Carlos Alberto Kalinovski; VARELA, Daniel de Andrade; SANTOS, Luciana Oliveira Penna. Metodologia de pesquisa em uso de inteligência artificial generativa: reflexões éticas e científicas na prática acadêmica. Annales FAJE, Belo Horizonte, v. 9, n. 5, p. 100-107, 2024. Disponível em: Disponível em: https://www.faje.edu.br/periodicos/index.php/annales/article/view/5946 Acesso em: 25 fev. 2025.
  » https://www.faje.edu.br/periodicos/index.php/annales/article/view/5946
• MACHADO, Ariél Philippi ; HOFFMANN, Carlos Alberto Kalinovski ; VARELA, Daniel de Andrade ; SANTOS, Luciana Oliveira Penna ; RAMOS, Alexandre Moraes. Estudo de tendências na pesquisa acadêmica utilizando inteligência artificial generativa: uma inspiração a partir do método Delphi. In: Anais do XXIII Congresso Internacional de Gestão Universitária - CIGU, Montevideo, 2024b. p. 1-16. Disponível em: Disponível em: https://repositorio.ufsc.br/bitstream/handle/123456789/261278/2024105.pdf Acesso em: 25 fev. 2025.
  » https://repositorio.ufsc.br/bitstream/handle/123456789/261278/2024105.pdf
• MAGALHÃES, Nara Maria Emanuelli; RODRIGUES, Cláudia Medianeira Cruz. A cultura de avaliação da Universidade Federal do Rio Grande do Sul (UFRGS): entre a avaliação externa e a autoavaliação. Ensaio: Avaliação e Políticas Públicas em Educação, Rio de Janeiro, v. 28, n. 109, p. 909-931, dez. 2020.
• MOROSINI, Marília (org.). Enciclopédia Brasileira de Educação Superior - EBES Porto Alegre: EDIPUCRS, 2021. v. 1.
• SOUSA, José Vieira de; GRIBOSKI, Claudia Maffini. Autoavaliação institucional no marco do SINAES: movimentos e tensões no uso dos resultados para a qualidade da educação superior. Revista Cocar, Edição Especial, n. 29, p. 1-20, 2024.
• Times Higher Education. Impact Rankings 2025: Methodology explained Disponível em: Disponível em: https://www.timeshighereducation.com/world-university-rankings/impact-rankings-2025-methodology Acesso em: 14 set 2025.
  » https://www.timeshighereducation.com/world-university-rankings/impact-rankings-2025-methodology
• UNESCO. Higher Education Global Data Report: Working document, May 2022 Disponível em: Disponível em: https://www.right-to-education.org/sites/right-to-education.org/files/resource-attachments/UNESCO_Higher%20Education%20Global%20Data%20Report_Working%20document_May2022_EN_0.pdf Acesso em: 14 set 2025.
  » https://www.right-to-education.org/sites/right-to-education.org/files/resource-attachments/UNESCO_Higher%20Education%20Global%20Data%20Report_Working%20document_May2022_EN_0.pdf
• UNIVERSIDADE DO EXTREMO SUL CATARINENSE (UNESC). Balanço social 2023: relatório de responsabilidade social Criciúma: Universidade do Extremo Sul Catarinense, 2023. Disponível em: Disponível em: https://www.unesc.net/portal/resources/official_documents/24733.pdf Acesso em: 17 mar. 2025.
  » https://www.unesc.net/portal/resources/official_documents/24733.pdf
• UNIVERSIDADE DO EXTREMO SUL CATARINENSE (UNESC). Plano de desenvolvimento institucional 2024-2028 Criciúma: Universidade do Extremo Sul Catarinense, 2024. Disponível em: Disponível em: https://www.unesc.net/portal/resources/official_documents/25068.pdf Acesso em: 17 mar. 2025.
  » https://www.unesc.net/portal/resources/official_documents/25068.pdf
• UNIVERSIDADE FEDERAL DO RIO GRANDE (FURG). Histórico da FURG Disponível em: Disponível em: https://www.furg.br/a-furg/historia Acesso em: 19 fev. 2025.
  » https://www.furg.br/a-furg/historia
• UNIVERSIDADE FEDERAL DO RIO GRANDE (FURG). Plano de desenvolvimento institucional 2024-2028 Rio Grande: FURG, 2024. Disponível em: http://www.pdi.furg.br. Acesso em: 17 mar. 2025.
• UNIVERSIDADE FEDERAL DO RIO GRANDE (FURG). Resolução nº 027/2016 - Regimento da Comissão Própria de Avaliação (CPA) Rio Grande: FURG, 2016. Disponível em: Disponível em: https://avaliacao.furg.br/images/CPA/Resolucoes/Resoluo_N_027_2016_-_CPA.pdf Acesso em: 19 mar. 2025.
  » https://avaliacao.furg.br/images/CPA/Resolucoes/Resoluo_N_027_2016_-_CPA.pdf

• 1
  Article published with funding from theConselho Nacional de Desenvolvimento Científico e Tecnológico- CNPq/Brazil for editing, layout and XML conversion services.
• 2
  FAVI: <https://drive.google.com/file/d/1MmSp7AiwEJICIdPwdYxzzEqtFb2foNFX/view>; FURG: <https://avaliacao.furg.br/relatorios-de-autoavaliacao-inep/ciclo-avaliativo-2018-2021>; UNESC: <https://www.unesc.net/files/editor/files/Rel_CPA_Final_2023_VF_Ficha_Catalografica.pdf>.
• DECLARATION OF DATA AVAILABILITY
  The underlying content of the research text is contained in the manuscript.

Edited by

• Section editor:
  Suzana dos Santos Gomes

Data availability

The underlying content of the research text is contained in the manuscript.

Publication Dates

• Publication in this collection
  15 Dec 2025
• Date of issue
  2025

History

• Received
  03 June 2025
• Preprint posted on
  21 May 2025
  10.1590/SciELOPreprints.11985
• Accepted
  17 Sept 2025