The ChatGPT: Revolutionizing Research with AI

Abstract

Artificial Intelligence is transforming Higher Education, achieving significant advances in different areas of knowledge. The OpenAI language model has revolutionized the world since November 2022. This study explores the potential of applying ChatGPT in academia. Documents available in the Web of Science, Scopus and Google Scholar are analyzed. The keywords “ChatGPT” and “Higher Education” are used using the “AND” operator, year 2023. The results of the bibliometric analysis reveal a considerable number of articles. When filters are applied, the number is significantly reduced. The subject is at an early stage in relation to HE. Publications in more than one database were identified, but not publication of an article in more than one journal; there is interest from researchers from different countries. Literature review studies are abundant. The advantages in terms of efficiency, quality and creativity in the generation of knowledge are highlighted. However, there are also ethical challenges and the need for responsible use. It is essential that educators and students are aware of the potential risks and take steps to mitigate them through ongoing training and the promotion of academic integrity.

Key words
ChatGPT; Higher Education; Artificial Intelligence; Scopus; Web of Science; Google Scholar

INTRODUCTION

Artificial Intelligence (AI) is an increasingly relevant technology in different fields (Sevilla University Library 2023, p. 5), including Higher Education (HS), and is generating a significant transformation in the way in which we relate to knowledge. Haugeland (2001) defines AI as those machines that have the ability to reason and think, while Boden (2017) describes it as the ability of computers to perform tasks that would normally require the intervention of human intelligence.

This technology has advanced significantly in recent years, and Rouhiainen (2018) describes it as the ability of machines to use algorithms, learn from data, and apply that knowledge in decision-making, thus mimicking the human thought process.

In the context of higher education, the use of AI, and in particular ChatGPT, has experienced rapid growth in the last 5 years, as pointed out by Chu et al. (2022) apud by Crompton & Burke (2023). The proliferation of new AI tools has had a significant impact on the way academic research is approached. Howell (2023) mentions that even the university has not escaped the arrival of ChatGPT and the spread of AI in general.

ChatGPT, developed by OpenAI, has demonstrated its ability to revolutionize the generation of knowledge in the academic and scientific fields; it has taken the world by storm, reaching 100 million users just two months after its launch (UNESCO 2023b p. 16). Since its launch in November 2022, this AI-based model has left a significant mark, especially with the ChatGPT-3.5 version and later with the ChatGPT-4 version in March 2023.

Prior to the release of the latest version of ChatGPT, the consulting firm Intelligent.com (2023) conducted a survey of 1,000 college students in the United States to inquire about their knowledge and use of ChatGPT in relation to school assignments. The results revealed that 30% of university students have used ChatGPT in their writing tasks. Within this group, approximately 60% use it for more than half of their tasks. Surprisingly, 3 out of 4 ChatGPT users admitted that they consider using it to constitute cheating, but despite this perception, they continue to use it.

AI has gradually been accepted by colleges and universities as an effective tool to automate a number of tasks effectively and efficiently. The article published by The Stanford Daily discusses how universities are using AI to improve student enrollment and retention and the debate it has sparked on campus about its role in academic integrity (Allen & Hochman 2023).

Following the same authors, Allen & Hochman (2023) state: Some argue that using ChatGPT to complete tasks is a violation of the Code of Honor, while others claim that it is simply a tool for generating ideas. Strzelecki (2023) refers: the advances support the idea that AI and in particular ChatGPT, has become a powerful ally to enhance the teaching-learning process, allowing greater efficiency in the production of knowledge.

However, this use of AI in HE poses challenges and risks associated with its use (Morales 2023) that must be considered. Some argue that over-reliance on tools like ChatGPT can affect human interaction and make it difficult to function without their help. Additionally, there are concerns about the presence of bias and the ethical implications of using AI in educational decision-making.

Along the same lines as Morales, the use of AI tools in HE presents a significant challenge and opportunity today. HE plays a crucial role in the development of individuals and societies, and the use of AI in this field has generated growing interest. However, this technological advance has also raised issues that require attention.

It is essential to understand the impacts, advantages and challenges associated with the use of AI tools in higher education, which can range from improvements in learning personalization and administrative efficiency to ethical considerations related to data privacy, equity of access and potential replacement of traditional jobs. In this context, it is essential to examine in detail the implementation of AI in Higher Education institutions, its influence on teachers and students, as well as the measures taken to ensure the quality of education and equity of access. It is also necessary to explore the ethical and legal challenges that may arise, as well as the impact on the skills needed for the workforce of the future.

This paper aims to contrast the results obtained from scientific publications on the use of ChatGPT in Higher Education by searching academic platforms such as Web of Science (WoS), Scopus and Google Scholar (GS), followed by a quantitative analysis. The objective is to provide bibliometric results related to the object of study of the present work.

It seeks to assess the application of AI in HE in the year 2023, identifying trends, areas of focus and interdisciplinary collaborations. The purpose is to provide an updated overview of the state of knowledge in this field, in order to guide future research and educational policies.

MATERIALS AND METHODS

The central point of the bibliometric study is the use of quantitative methods, in order to carry out analysis objectively on a desired aspect (Abritta et al. 2023). For this study, a bibliometric analysis was carried out, which constitutes a methodology that allows the quantitative evaluation and analysis of scientific production in a certain field. In accordance with the objective set for the development of the study, a literature review was carried out using secondary sources of information consisting of scientific articles.

The summaries taken from the Elsevier solutions were evaluated: WoS, and Scopus; plus, the GS platform. In the choice of “the three bibliometric indicators” (Neto 2022) we relied on the study by Martín et al. (2018) who highlighted the lack of evidence on the differences between GS, WoS and Scopus in terms of research query and evaluation. Also, in studies that have used the same databases (Araújo & Souza 2023, Pessanha et al. 2023, Teixeira et al. 2023)

Furthermore, the inclusion of GS in our selection is supported by its inclusive approach in crawling the web and its extensive coverage, especially in disciplines such as Humanities and Social Sciences where WoS and Scopus are known to be weak (Chavarro et al. 2018, Mongeon & Paul-Hus 2016, Van Leeuwen et al. 2001, Buchinger et al. 2014 cited in Martín et al. 2018 ) and avoided, databases that is mainly oriented to the medical field and biomedical sciences such as PubMed (Buchinger et al. 2014).

This diversity of academic sources enriches the understanding of how ChatGPT is used in different academic contexts within HE.

The equation “(chatGPT) AND (higher education)” adapted from Garcés et al. (2022) has been used. The results have been refined depending on the options provided by the platform; items match:

- Year of publication: 2023

- Document Type: Articles

The Elsevier solutions (WoS and Scopus) were accessed through the CICCO Portal of the National Council of Science and Technology CONACYT of Paraguay. GS was accessed through Google (Table I). The same table shows the differences using only the equation and then the results applying the filters.

Table I
Sample identification.

Analysis unit	Access through	Total without filters	Total with filters
Web of Science	CICCO Portal	32	31
Scopus	CICCO Portal	30	23
Academic google	Google	16900	133
		16962	187

• Number of posts in 2023 that contain the words ChatGPT and higher education. Data taken as of July 1, 2023.

For the GS, the formula shown in Figure 1 was also applied; this search yielded 16900 results; then the following filters were used: year of publication and type of document articles; the result yielded 133 articles in total.

Figure 1
Search result in Google Scholar using the equation without the preset filters.

Before refining the WoS search the result is 32 articles (Figure 2). Subsequently, the filters are applied and there is a small variation from 32 to 31 articles. On the other hand, when using the preset filters in Scopus, the number drops considerably; from 30 articles to 23 articles (Figure 3).

Figure 2
Search result in WoS using the equation without the preset filters.

Figure 3
Search result in Scopus using the equation without the preset filters.

In such a way to facilitate the analysis, a new filter was applied with the keywords: ChatGPT, Higher Education, Artificial Intelligence, option provided by Scopus. WoS and GS do not have the option to refine the results by keywords, it was decided to take the first 10 articles that were already ordered by relevance and then proceed to analyze the results of the abstracts.

The units of analysis used were adapted from Castillo & Carreton (2010).

Analysis unit 1: Production level, whose variables are: number of publications, gender of authors, countries.

Unit of analysis 2: Dispersion indicators whose variable is the number of citations per article.

Analysis unit 3: Level of use of scientific literature. The variables are the publication of an article in more than one journal.

Analysis unit 4: Level of empirical rigor focused on the Type of research. Sample.

Analysis unit 5: Level of results, where the conclusions reached constitute the variables to be analyzed. Whether the use of GPT chat is recommended or not in the ES. (Exclusive)

As no information was found in the results that reveals the level of acceptance and adoption of ChatGPT in the academic field or in scientific production, the articles have been discarded. 13 Scopus abstracts were analyzed; 15 abstracts from WoS and 20 abstracts from GS were analyzed for each platform, selecting 10 articles that met the pre- established requirements.

RESULTS AND DISCUSSION

In this section, the relevant results that emerged are clearly and concisely presented. These results will provide a detailed and objective view of the data obtained by exploring the potential use of ChatGPT in academic research in the field of higher education.

According to the review for Analysis Unit 1, the following results could be extracted

In this Unit, the Production Level is measured through the Variable “Number of publications applying filters” (Table II), data has been collected and analyzed from three selected databases. The results show that a total of 187 relevant posts were obtained after applying the preset filters. It is interesting to note that GS is the source with the highest number of publications (133), which suggests that it is a widely used tool in the academic and scientific field to access scientific information.

Table II
Analysis unit 1. Production level. Variable: Number of publications applying filters.

Units of analysis	Number
Web of Science	31
Scopus	23
Academic google	133
Total	187

• Data taken as of July 1, 2023.

On the other hand, WoS and Scopus “the great allies of every researcher” (Pérez 2017) have also provided a significant number of publications (31 and 23, respectively). These databases are recognized for their rigor and precision in the collection of academic and scientific information, which indicates that they continue to be reliable sources for obtaining relevant research papers.

The difference in the number of publications between the databases may be due to several factors, such as the coverage of journals on each platform, the reach of their indexes, and accessibility for researchers. It is important to take these differences into account when interpreting the results and consider the combination of several sources to obtain a more complete vision of the scientific production in the study area.

In order to confirm the theory of Martín et al. to the; added the keyword “Elsevier” to the equation, returning 152 Elsevier post results containing the word ChatGPT. This search was done only once to verify the theory Martín et al. (2018). The search for the article called: The false positives and false negatives of generative AI detection tools in education and academic research: The case of ChatGPT available both in Scopus and in WoS was also carried out. The result was negative, so it can be said that not all Elsevier articles are indexed in the GS platform, that is, the overlap is not complete.

In Analysis Unit 1, where the Production Level is measured using the Variable “Gender of the authors” (Table III), the results reveal the predominance of publications with male authors (72) over publications by women (33).

Table III
Analysis unit 1. Production level. Authors’ gender.

Units of analysis	Female	Male
Web of Science	10	23
Scopus	8	20
Academic google	21	41
	39 - 6 = 33	84 – 12 = 72

• The corresponding discount has been made due to overlapping items.

When comparing the gender participation between the databases, we can notice that in WoS and Scopus, there is a higher representation of male authors compared to female authors. However, in GS, the difference between the genres is less pronounced, with a more balanced number of publications between male and female authors.

It is interesting to note that a discount was made in the total number of authors due to the overlap of articles between the WoS and Scopus solutions that will be explained in analysis unit 3, but this discount was not applied in GS because it was not identified overlap.

These results may suggest the existence of gender inequalities in scientific production, especially in certain databases. The gender gap found in WoS and Scopus may raise questions about the possible barriers that women face in the academic and scientific field to publish their research.

According to the study by Cheryan, apud Farias & Oliveira (2018), women obtain almost half of university degrees in biology, chemistry and mathematics, but less than 20% in Computer Science, Engineering and Physics.

Battirola et al. (2022) adds:

Globally, data reveals that despite the increase in the number of researchers, stark gender inequality persists in STEM fields. In the specific case of Brazil, in terms of gender equality in science, it is behind Portugal (58%) and Argentina (51%), but ahead of the United States (33%) and Germany (32%). These values cover all areas of knowledge, there is a notable gender gap in the exact sciences, where the number of female researchers is considerably lower. The percentage of women enrolled in STEM disciplines in Latin America is 37%, compared to men (p.1).

In the latest publication made by UNESCO (2023a) in remembrance of the International Day of Girls and Women in Science, statistical data from 2017 are highlighted: where the percentage of women in the 20 main countries for the proportion of professionals with AI skills, are: Singapore, South Africa and Italy with 28%, Turkey, Australia and Canada 24%, United States with 23%, followed by India with 22%, France and the Netherlands 21%; Sweden and the United Kingdom 20%, Switzerland and Spain 19%, Argentina 17%; Germany and Poland 16%; The last places are occupied by Mexico with 15% and Brazil with 14%.

In total there are 25 researchers distributed in countries such as Singapore (1), South Africa (1), Turkey (1), Australia (6), United Kingdom (7), United States (2) and Germany (8), results that coincide with in countries with the UNESCO list. The other researchers are from Nigeria (2), New Zealand (1), Ghana (1), England (1) and Kuwait (1).

In Analysis Unit 1, where the Level of Production is evaluated according to the “Countries of the authors”, various investigations from different countries have been identified. Next, relevant aspects of the results obtained will be discussed:

The results show a diverse representation of countries, which indicates that the scientific production related to the topic studied is not limited to a specific region. Countries from different continents, such as Australia, China, the United States, England, Singapore, South Africa and others, are already contributing relevant research in the area. The geographical distribution is quite considerable.

Two works have been the result of collaborations between researchers from different countries. For example, a joint investigation between researchers from England and Cambodia, as well as other collaborations between researchers from Qatar and Tunisia.

This suggests that cooperation between countries is an increasingly common trend in academic research.

Although the sample size is limited, it is possible to identify certain emerging trends in AI scientific production. By way of example: in addition to research from countries recognized for their leadership in research, such as the United States or the United Kingdom, there is also the participation of emerging countries such as Bangladesh, the United Arab Emirates, Ghana, Nigeria and Turkey. This demonstrates that research in AI and related technologies is not limited to developed nations, and that opportunities exist for developing countries to significantly contribute to the advancement of knowledge in the field.

It should be mentioned that the amount of research in some countries may be influenced by the size of the sample. Only one investigation is registered in the sample, such as Germany, Kuwait, Jordan, New Zealand, the Netherlands, Poland, Thailand, among others. Countries with a higher representation in the sample (Australia) may show a greater number of papers, but this does not necessarily reflect their leadership in research in ChatGPT and SE and the others reflect the potential for research growth on the subject.

With only 24 articles, AI-relevant countries are likely not represented in this sample. It is necessary to consider that there is a wide field of research in AI worldwide, and the results obtained here are only a small sample of the total scientific production. It is recalled that of the 30 selected articles, 6 were discarded because they were duplicated in WoS and Scopus.

Analysis unit 2: Dispersion indicators whose variable is the number of citations per article.

Cuevas et al. (2013) mention: “The number of citations is a widely used indicator for the evaluation of works published in academic journals and their authors”. In this unit, the dispersion indicators based on the “Number of citations” (Table IV) of the selected articles have been taken into account. The results show significant variability in the number of citations received for each article, indicating different levels of relevance and recognition within the academic community.

Table IV
Analysis unit 2. Dispersion indicators. Number of appointments.

Articles	Quotes
Learning with ChatGPT 3.5 as a more knowledgeable other: an autoethnographic study.	0
ChatGPT: An ever-increasing engagement of artificial intelligence in online assessment in distance education.	53
The false positives and false negatives of generative AI detection tools in education and academic research: The case of ChatGPT.	0
Holy or Unholy? Interview with Open AI’s ChatGPT.	11
The effect of generative artificial intelligence (AI)-based tool use on students’ computational thinking skills, programming self-efficacy and motivation	0
Prompting Higher Education Towards AI-Augmented Teaching and Learning Practice	2
ChatGPT versus engineering education assessment: a multidisciplinary and multi- institutional benchmarking and analysis of this generative artificial intelligence tool to investigate assessment integrity	0
To use or not to use ChatGPT in higher education? A study of students’ acceptance and use of technology	0
From human writing to artificial intelligence generated text: examining the prospects and potential threats of ChatGPT in academic writing	5
A SWOT analysis of ChatGPT: Implications for educational practice and research	6
It’s time to review the existing student performance assessment approach Higher Education in a New Era of ChatGPT- A case study	0
“Knock, Knock... Who’s There?” ChatGPT and Artificial Intelligence-Powered Large Language Models: Reflections on Potential Impacts Within Health and Physical Education Teacher Education …	0
Leadership Needed for Ethical ChatGPT: Character, Assessment and Learning Using Artificial Intelligence (AI)	2
Chatting and cheating: ensuring academic integrity in the age of ChatGPT.	9
ChatGPT: Talk about nonsense or the end of traditional evaluations in higher education?	170
Exploring the Opportunities and Challenges of NLP Models in Higher Education: Is Chat GPT a Blessing or a Curse?	2
What is the impact of ChatGPT on education? A quick review of the literature	23
ChatGPT for education and research: a review of benefits and risks.	10
ChatGPT for teaching, learning and research: perspectives and challenges.	9
ChatGPT forever? On the opportunities and challenges of the great linguistic models for education.	156
ChatGPT as an educational tool: opportunities, challenges and recommendations for courses in communication, business writing and composition	17
Education in the age of generative artificial intelligence (AI): understanding the potential benefits of ChatGPT to promote teaching and learning	120
ChatGPT and academic research: a review and recommendations based on practical examples	10
So what if ChatGPT wrote it?” Multidisciplinary perspectives on opportunities, challenges, and implications of generative conversational AI for research, practice, and policy.	149

• Data taken as of July 1, 2023.

Although researchers frequently consult citation counts provided by Google Scholar (GS), Web of Science (WoS), and Scopus, and sometimes use them in research assessments, there is no recent or systematic evidence on disparities between they; In addition, they consider GS as a “superset” (Martín et al. 2018) that brings together a wide variety of academic and scientific resources.

Although it is not a database in the traditional sense like WoS or Scopus, GS collects and displays relevant bibliographic information and allows specific searches in the academic field. For example, when searching in GS, with the equation

$$\text{(chatGPT) AND (higher education)}$$

they were found among articles from the scientific publisher Elsevier. Articles from Wiley Online Library, Taylor & Francis, Springer, arxiv.org among others are also observed.

When analyzing the data, it is observed that several articles have received a low number of citations, even some of them have not been cited so far. This may be due to different factors, such as the novelty of the topic addressed, the availability and accessibility of the articles, or the lack of adequate dissemination by the authors.

On the other hand, there are some articles that have been widely cited, reaching a considerable number of citations. These works seem to have generated a greater impact and attraction within the field of study, which may be due to the relevance of their research, the quality of the findings, or the relevance of their conclusions to the scientific and educational community. The most cited are from Singapore (170), Germany (156), the United Kingdom (149) and Ghana (120); all of the extracted from the GS platform.

It should also be noted that most of the selected articles are recent, which indicates that the topic of the use of ChatGPT in higher education is an emerging line of research. The publication date of these works reflects a growing interest in the field of AI applied to education and how ChatGPT can be a relevant tool in this context.

Being a relatively new topic, it is understandable that some of these articles have not yet received a high number of citations. The academic and scientific community often requires time to evaluate and assimilate new concepts and approaches. However, the presence of some articles with a significant number of citations suggests that the topic has caught the attention of researchers and professionals in a short period of time.

Analysis unit 3: Level of use of scientific literature. The variables are the publication of an article in more than one journal.

In Analysis Unit 3, the issue of “Article Overlap” in two widely used academic databases: WoS and Scopus from Elsevier publishing house is addressed. Of the articles selected for this study, it is observed that some of them are present in both databases, but not in more than one scientific journal.

The appearance of articles between WoS and Scopus may be due to several factors, such as the indexing policy of both databases, the lack of synchronization in updating content or the inclusion of different journals and academic sources in each platform.

Yes, it is a common fact that many academic databases share similar information and content, which can lead to duplication of references or works in different media. This situation may be due to several reasons, such as the interconnection of different databases, cross-indexing or even human errors when entering information.

The analyses of the articles reviewed did not reveal duplicate publications. However, it is crucial to emphasize that duplication of scholarly work can have significant ethical and academic consequences. This could artificially inflate research results and distort the actual scientific output. Therefore, researchers should be fully aware of the importance of maintaining academic integrity (Killian et al. 2023) and avoid simultaneous publication in multiple journals. Rode & Lopes (2022) emphasize this precaution: “Authors should not submit the same manuscript to more than one journal, even in different languages, and should avoid duplicate publications, even when substantial parts are identical” (p. 70).

Repeated importation of the same reference inadvertently is a major concern in the academic community, as it can lead to overestimation of scientific production and distort bibliometric analysis. Additionally, this duplication can affect the accuracy and reliability of impact and visibility metrics for researchers and their work.

To avoid inadvertent duplication (Table V), it is critical that researchers be aware of this possibility and take steps to properly check and clean their reference lists before importing them into their papers. Using bibliographic management tools and carefully comparing references can help to avoid unintentional duplication.

Table V
Analysis unit 3. Dispersion indicators. Overlapping items.

Articles	Quotes
	Scopus	WoS
Learning with ChatGPT3.5 as a more informed other: an autoethnographic study	0	0
ChatGPT: A growing invasion of artificial intelligence in online distance education assessment.	0	53
The false positives and false negatives of generative AI detection tools in education and academic research: The case of ChatGPT.	0	0
Holy or wicked? Interview with Open AI? ChatGPT?	6	5
The effect of using generative artificial intelligence (AI)-based tools on computational thinking skills, programming self-efficacy, and student motivation	0	0
Driving higher education towards AI-augmented teaching and learning practice	2	0
ChatGPT versus engineering education and assessment: a multi-disciplinary, multi- institutional benchmarking and analysis of this generative artificial intelligence tool to investigate assessment integrity	0	0
To use or not to use ChatGPT in higher education? A study on the acceptance and use of technology by students.	0	0
From human writing to AI-generated text: examining the prospects and potential threats of ChatGPT in academic writing	3	2
A SWOT Analysis of ChatGPT: Implications for Educational Practice and Research	4	2

• Data taken as of July 1, 2023.

The problem of plagiarism in HE is a complex and constantly evolving issue. As highlighted by Novo et al. (2020), this phenomenon undermines the integrity of the academic process and ultimately diminishes the value of higher education. The emergence of AI tools such as ChatGPT adds a new dimension to this challenge. As Oliveira cited in Lima (2023) points out, the similarity between ChatGPT generated narrative and human writing make plagiarism detection even more complicated (p.3). The recommendation to maintain integrity is to properly cite the source (GPT) when employing its content generation in your work. This is ethical and respects copyright (Atlas 2023, p. 31).

In this context, it is essential not only to rely on existing plagiarism detection tools, but also to foster a culture of academic integrity from the very beginning of higher education. Students must understand the importance of originality in their work and the value of constructing their knowledge in an ethical manner. In addition, educators have a responsibility to design assessments that encourage creativity and reflection, thereby reducing opportunities for plagiarism.

On the other hand, academic institutions must continue to invest in the development of more advanced plagiarism detection technologies that can identify AI-generated work.

However, this should not be limited to mere prosecution of plagiarism, but should be accompanied by an emphasis on education and prevention.

Analysis unit 4: Level of empirical rigor that focuses on the Type of research. Sample.

Based on the types of research and samples mentioned in the articles related to the use of ChatGPT in higher education, we can see that most of them have opted for theoretical analysis and literature review approaches. These studies have focused on understanding and evaluating the capabilities of ChatGPT in the educational context, especially in terms of assessment, learning, and teaching.

The research of an exploratory and descriptive nature has allowed obtaining a broader and more general vision of the possible applications of ChatGPT in higher education. Literature review and experiments with AI software have been key tools to analyze how this model of natural language processing can be effectively integrated into teaching and learning (Jürgen et al. 2023).

The analytical approaches adopted in some articles have delved into the challenges, opportunities, benefits and risks related to the use of ChatGPT in education and research. This analytical perspective has made it possible to identify specific aspects that can influence the adoption and successful application of this technology in the educational field.

Regarding the samples used in the studies, it is interesting to note that some articles do not provide explicit details about them, which suggests that they have relied more on literature review and theoretical analysis. Instead, other articles have used more specific samples, such as 50 relevant articles from databases and GS, to analyze the performance and application of ChatGPT in different subject domains.

On the other hand, Strzelecki (2023) has already worked with students. Yilmaz & Karaoglan (2023), in their experimental study with students who used ChatGPT in programming practices, were able to observe that they improved significantly more in computational thinking skills, programming self-efficacy, and motivation towards the lesson compared to students in the control group who used.

It is relevant to note that most of the articles focus on general aspects of the use of ChatGPT in higher education, without specifying particular samples of subjects or participants. This may be due to the novelty of the topic and the recent appearance of studies on this specific topic.

Unit of analysis 5: Level of results, where the conclusions reached constitute the variables to be analyzed. Whether the use of GPT chat is recommended or not in the ES. (Exclusive)

This unit focuses on the “Level of results and the conclusions reached in the different articles on the use of ChatGPT in HE”. From the information analyzed, it is evident that researchers have identified both strengths and limitations (Jürgen et al. 2023) in the educational field when using ChatGPT and other AI tools.

In terms of benefits, it has been noted that ChatGPT can provide personalized learning and on-demand support to students. The ability to generate personalized learning plans (Fuchs 2023, Baidoo & Owusu Ansah 2023), provide feedback (Fuchs 2023), and offer resources in a timely and convenient manner can be especially valuable in the educational context. Likewise, its ability to improve pedagogical practice (Sok & Heng 2023) and offer personalized virtual tutoring has been highlighted, which can improve (Yogesh et al. 2023) the educational experience (Opara et al. 2023) of students.

However, challenges (Fuchs 2023, Opara et al. 2023) and risks associated with the use of ChatGPT have also been identified (Sok & Heng 2023, Yogesh et al. 2023, Kasneci et al. 2023) in higher education. Among them, the possible loss of human interaction is mentioned (Fuchs 2023, Kasneci et al. 2023, Lo 2023), the presence of biases (Kasneci et al. 2023) and ethical issues (Fuchs 2023, Naidu & Sevnarayan 2023). Furthermore, it has been observed that the performance of ChatGPT varies across subject domains (Lo 2023), showing outstanding performance in some domains and less satisfactory performance in others.

It is noteworthy that some researchers suggest recommendations to address these challenges and reap the benefits of ChatGPT in a responsible manner (Rahman et al. 2023, Sok & Heng 2023, Kasneci 2023, Baidoo & Owusu Ansah 2023). For example, it is proposed to properly cite and reference the answers provided by ChatGPT to guarantee the quality and credibility of the information generated by this technology in the educational context (Dergaa et al 2023) and academic integrity (Crawford 2023, Cotton et al. 2023, Killian et al. 2023).

Studies propose and develop a model to examine the factors that predict the adoption and use of ChatGPT among higher education students (Strzelecki 2023, Crawfor et al. 2023, Iffat et al. 2023, Cotton et al. 2023, Kasneci et al. 2023, Mizanur et al. 2023, Fuchs 2023, Lo 2023, Opara et al. 2023, Stojanov 2023, Naidu & Sevnarayan 2023). Regarding the role of ChatGPT in higher education, it was pointed out that it can help instructors to delegate monotonous tasks such as grading and allow them to focus on more intellectual tasks. Students can also use ChatGPT to generate ideas. However, it was warned that excessive use of this technology could decrease students’ critical thinking (Kasneci et al. 2023) and increase educational inequalities.

It is argued that both teachers and students must develop the necessary skills and literacies to understand technology and its limitations (Kasneci et al. 2023, Jürgen et al. 2023). The importance of critical thinking and fact-checking strategies is emphasized to make the most of great language models in the educational setting. AI chatbots technology like ChatGPT could influence the future of learning, teaching and assessment in higher education but it will also help in the teaching-learning process (Jürgen et al. 2023).

Despite the limitations and challenges, it was concluded that generative pretrained transformers such as ChatGPT remain promising tools for communication and information retrieval in the higher education setting. The importance of addressing concerns about false detection of AI-generated text (Dalalah & Dalalah 2023) and ensuring ethical and responsible use of these models was emphasized.

In the academic context, it was emphasized that ChatGPT cannot replace human and intellectual creativity (Iskender 2023, Yilmaz & Karaoglan 2023, Eager & Brunton 2023, Nikolic et al. 2023), since originality and novelty are lacking in the products generated by the IA (Eager & Brunton 2023, Naidu & Sevnarayan 2023, Yilmaz & Karaoglan 2023).

In a specific study on teaching programming, it was found that the use of AI technologies such as ChatGPT significantly improved computational thinking skills (Yilmaz & Karaoglan 2023), programming self-efficacy, and student motivation compared to the control group. This suggests that the use of AI technologies can be beneficial in teaching programming and improving learning processes in higher education.

The articles also underscore the importance of institutions ensuring that their use of AI aligns with their values and mission, and that students are informed about how their data is used. It was noted that AI is not a one-size-fits-all solution to all challenges in HE, but one more tool that can be used to achieve innovative results (Strzelecki 2023) in education and research without straying from ethical aspects (Strzelecki 2023, Dalalah & Dalalah 2023).

The authors agree that ChatGPT is transforming teaching and learning in HE by enabling a more personalized experience and improving retention and effectiveness of online learning. Institutions that harness the power of this technology will achieve significantly better learning outcomes and provide a high-quality educational experience for their students.

CONCLUSIONS

When considering the five units of analysis addressed in the articles studied on the use of ChatGPT in HE, several important findings can be highlighted. However, it is important to remember that these results should be interpreted with caution due to the limited sample size and the various considerations discussed in the review.

In the first unit of analysis, a geographic diversity in scientific production is observed, with a broad representation of countries and international collaborations. This underscores the importance of global cooperation in AI research for higher education. Variability in gender representation across the different databases used in this research is also identified, highlighting the continued need to address and mitigate gender inequalities in science.

In the second unit of analysis, the visibility and impact of articles is found to vary considerably, underscoring the need to promote greater dissemination and promotion of research in this emerging area. As more researchers become interested and address this topic, it is likely that new advances and discoveries will be generated that will contribute to the development of AI applied to education and benefit the learning process in higher education.

In the fourth unit of analysis, studies on the use of ChatGPT in HE is characterized by an exploratory and descriptive approach, with literature review. This suggests the need for more detailed research to fully understand the applications and capabilities of this technology in the educational setting.

Finally, in the fifth unit of analysis, it is highlighted that it is advisable to use ChatGPT in HE with a clear understanding of its strengths and limitations. Educators and students should be aware of the potential risks and take steps to mitigate them, while taking advantage of its benefits in enhancing learning and teaching.

Finally, the fifth unit of analysis highlights that it is advisable to use ChatGPT in higher education with a clear understanding of its strengths and limitations. Educators and students should be aware of the potential risks and take steps to mitigate them, while taking advantage of its benefits in enhancing learning and teaching.

AI has experienced rapid growth since its emergence in the 1950s and is increasingly present in a variety of fields, including education. Acquiring skills to accompany the training of students in an ever-evolving technological world is critical. Integrating ChatGPT into education can improve teaching and personalize learning, preparing students for the future and responding to the complexities of human language for the benefit of society.

ACKNOWLEDGMENTS

The authors declare that they have no competing financial interests or known personal relationships that might have influenced the work reported in this paper.

REFERENCES

• ABRITTA MLR, MEIRA MGJ, FREITAS DA, MONTEIRO JRS & SOARES WD. 2023. Thematic and bibliometric analysis of publications in national physical education journals. Brazilian Mag Funct Health 11(1): 34. https://doi.org/10.25194/rebrasf.v11i1.1566
• ALAFNAN MA, SAMIRA D, MARINA J & KOBA L. 2023. ChatGPT as an Educational Tool: Opportunities, Challenges, and Recommendations for Communication, Business Writing, and Composition Courses. J Artific Intel Technol 3(2): 60-68. https://doi.org/10.37965/jait.2023.0184.
  » https://doi.org/10.37965/jait.2023.0184
• ALLEN CM & HOCHMAN S. 2023. Scores of Stanford students used ChatGPT on final exams, survey suggestions. The Daily Stanford. Available in https://stanforddaily.com/2023/01/22/scores-of-stanford-students-used-chatgpt-on-final-exams-survey-suggests/
  » https://stanforddaily.com/2023/01/22/scores-of-stanford-students-used-chatgpt-on-final-exams-survey-suggests/
• ATLAS E. 2023. ChatGPT for Higher Education and Professional Development: A Guide to Conversational AI. Accessed on: https://digitalcommons.uri.edu/cba_facpubs/548
  » https://digitalcommons.uri.edu/cba_facpubs/548
• ARAÚJO BJM & SOUZA JA. 2023. Professional standards as a factor in adapting human resources to industry 4.0: approaches to development and implementation in public administration. Brazilian J Product Eng 9(3): 114-131. https://doi.org/10.47456/bjpe.v9i3.41035.
  » https://doi.org/10.47456/bjpe.v9i3.41035
• BAIDOO-ANU D & OWUSU ANSAH L. 2023. Education in the Era of Generative Artificial Intelligence (AI): Understanding the Potential Benefits of ChatGPT in Promoting Teaching and Learning (January 25, 2023). Available at SSRN: http://dx.doi.org/10.2139/ssrn.4337484
  » http://dx.doi.org/10.2139/ssrn.4337484
• BATTIROLA LC, DE OLIVEIRA SG, LINARES DR, DE ALMEIDA BS, ULIANA MP & GONÇALVES CDCS. 2022. Gender parity in science: an analysis of the participation of women in Argentina, not Brazil and Paraguay. Virt J Chem 14.2. https://rvq-sub.sbq.org.br/index.php/rvq/article/view/4162
  » https://rvq-sub.sbq.org.br/index.php/rvq/article/view/4162
• BODEN MA. 2017. Artificial intelligence. Turner Publicaciones S.L. Primera edición. Madrid. Available in https://books.google.es/books?hl=es&lr=&id=LCnYDwAAQBAJ&oi=fnd&pg=PT3&dq=inteligencia+artificial+&ots=drXtxXcMhc&sig=VXN5qUeejISo264zt9WupeAuPco.
  » https://books.google.es/books?hl=es&lr=&id=LCnYDwAAQBAJ&oi=fnd&pg=PT3&dq=inteligencia+artificial+&ots=drXtxXcMhc&sig=VXN5qUeejISo264zt9WupeAuPco
• BUCHINGER D, CAVALCANTI GAS & HOUNSELL MS. 2014. Academic search engines: a quantitative analysis. Braz J Appl Comput 6(1): 108-120. https://doi.org/10.5335/rbca.2014.3452.
  » https://doi.org/10.5335/rbca.2014.3452
• CASTILLO A & CARRETON MC. 2010. Research in Communication. Bibliometric Study in Communication Magazines in Spain. Commun Soc 23(2): 289-327. https://doi.org/10.15581/003.23.36234.
  » https://doi.org/10.15581/003.23.36234
• CRAWFORD J, COWLING M & ALLEN K. 2023. Leadership is needed for ethical ChatGPT: Character, assessment, and learning using artificial intelligence (AI). J Univ Teach Learn Pract 20(3): https://doi.org/10.53761/1.20.3.02.
  » https://doi.org/10.53761/1.20.3.02
• CROMPTON H & BURKE D. 2023. Artificial intelligence in higher education: the state of the field. Int J Educ Technol High Educ 20: 22. https://doi.org/10.1186/s41239- 023-00392-8.
  » https://doi.org/10.1186/s41239- 023-00392-8
• CUEVAS SO, RANGEL SM & HUERTA LL. 2013. Overlap between Web of Science and Scopus citations: an exploratory study. Ibersid Inf Doc Syst Mag 7: 131-135. https://doi.org/10.54886/ibersid.v7i0.4080.
  » https://doi.org/10.54886/ibersid.v7i0.4080
• COTTON RD, COTTON AP & SHIPWAY JR. 2023. Chatting and cheating: Ensuring academic integrity in the era of ChatGPT, Innovations in Education and Teaching International. https://login.ezproxy-cicco.conacyt.gov.py/login?qurl=https://doi.org%2f10.1080%2f14703297.2023.2190148
  » https://login.ezproxy-cicco.conacyt.gov.py/login?qurl=https://doi.org%2f10.1080%2f14703297.2023.2190148
• CHAVARRO D, RÀFOLS I & TANG P. 2018. To what extent is inclusion in Web of Science an indicator of journal “quality”? Res Eval 27(2): 106-118. https://doi.org/10.1093/reseval/rvy015.
  » https://doi.org/10.1093/reseval/rvy015
• CHU Y, MEI J & XIE J. 2023. Exploring the effects of lipid oxidation and free fatty acids on the development of volatile compounds in grouper during cold storage based on multivariate analysis, Food Chemistry X 20: 100968. https://doi.org/10.1016/j.fochx.2023.100968
• DALALAH D & DALALAH O. 2023. The false positives and false negatives of generative AI detection tools in education and academic research: The case of ChatGPT. Int J Manag Educ 21(2): 100822. https://www.scopus.com/record/display.uri?eid=2-s2.0- 85160030074&doi=10.1016%2fj.ijme.2023.100822&origin=inward&txGid=b47e555bc 7af2dc8ac7097478e54b7e0.
• DERGAA I, CHAMARI K, ZMIJEWSKI P & SAAD H. 2023. From human writing to artificial intelligence generated text: examining the prospects and potential threats of ChatGPT in academic writing. Ben Biology of Sport 40(2): 615-6222023. https://doi.org/10.5114/biolsport.2023.125623.
  » https://doi.org/10.5114/biolsport.2023.125623
• EAGER B & BRUNTON R. 2023. Driving higher education towards AI-augmented teaching and learning practice. J Univ Teach Learn Pract 20(5): https://doi.org/10.53761/1.20.5.02.
  » https://doi.org/10.53761/1.20.5.02
• FARIAS SS & OLIVEIRA AD. 2018. Female Invisibility and Social Representations of Gender in Technology and Science. In 12th National Congress of Health Psychology: Promoting and Innovating in Health Psychology, no. 739, p. 731. Higher Institute of Applied Psychology. Available in https://repositorio.ispa.pt/bitstream/10400.12/6246/1/12CongNacSaude_731.pdf
  » https://repositorio.ispa.pt/bitstream/10400.12/6246/1/12CongNacSaude_731.pdf
• FUCHS K. 2023. Exploring the opportunities and challenges of NLP models in higher education: is Chat GPT a blessing or a curse? Front Educ 8: 1166682. https://doi.org/10.3389/feduc.2023.1166682.
  » https://doi.org/10.3389/feduc.2023.1166682
• GARCÉS GLF ET AL. 2022. Use of artificial intelligence in information management: a bibliometric review. Iber J Inf Syst Technol E54: 506-517. Available in https://www.proquest.com/openview/cd91b567d2231af1ab3b1838a62b3948/1.pdf?pq- origsite=gscholar&cbl=1006393.
• HAUGELAND J. 2001. Artificial intelligence. XXI century. Available in https://books.google.es/books?hl=es&lr=&id=BcKGEg_HBvYC&oi=fnd&pg=PA4&dq=inteligencia+artificial+&ots=fqQ7Fr3M5G&sig=3uiegk4IYNXaJTQ-xL4EnDhkbMI.
  » https://books.google.es/books?hl=es&lr=&id=BcKGEg_HBvYC&oi=fnd&pg=PA4&dq=inteligencia+artificial+&ots=fqQ7Fr3M5G&sig=3uiegk4IYNXaJTQ-xL4EnDhkbMI
• HOWELL CW. 2023. ChatGPT and the existential crisis of the university. Available in https://legrandcontinent.eu/es/2023/06/23/chatgpt-y-la-crisis-existencial-de-la- universidad/
  » https://legrandcontinent.eu/es/2023/06/23/chatgpt-y-la-crisis-existencial-de-la- universidad/
• IFFAT SCH, SAYED AAS, GHALEB AER & HABIB CH. 2023. Time to Revisit Existing Student’s Performance Evaluation Approach in Higher Education Sector in a New Era of ChatGPT — A Case Study. Cogent Educ 10: 1. https://doi.org/10.1080/2331186X.2023.2210461.
  » https://doi.org/10.1080/2331186X.2023.2210461
• INTELLIGENT.COM. 2023. Available in https://www.intelligent.com/nearly-1-in-3-college-students-have-used-chatgpt-on-written-assignments/
  » https://www.intelligent.com/nearly-1-in-3-college-students-have-used-chatgpt-on-written-assignments/
• ISKENDER A. 2023. Holy or impious? Interview with Open AI’s ChatGPT. European J Tour Res 34: 3414. https://doi.org/10.54055/ejtr.v34i.3169.
  » https://doi.org/10.54055/ejtr.v34i.3169
• JÜRGEN R, SAMSON T & SHANNON T. 2023. ChatGPT: Bullshit spewer or the end of traditional assessments in higher education? ED-TECH REVIEWS 6(1). Available in https://doi.org/10.37074/jalt.2023.6.1.9.
  » https://doi.org/10.37074/jalt.2023.6.1.9
• KASNECI E ET AL. 2023. ChatGPT for good? On opportunities and challenges of large language models for education. Learn Indiv Diff 103: 102274. https://doi.org/10.1016/j.lindif.2023.102274.
  » https://doi.org/10.1016/j.lindif.2023.102274
• KILLIAN CM, MARTTINEN R, HOWLEY D, SARGENT J & JONES EM. 2023. Knock, Knock…Who’s There? ChatGPT and Artificial Intelligence-Powered Large Language Models: Reflections on Potential Impacts Within Health and Physical Education Teacher Education. J Teach Phys Educ 42(3): 385-389. https://doi.org/10.1123/jtpe.2023-0058.
  » https://doi.org/10.1123/jtpe.2023-0058
• LIMA J. 2023. How ChatGPT affects education and university development. The Trends Hub, Porto, n. 3. https://doi.org/10.34630/tth.vi3.5020
  » https://doi.org/10.34630/tth.vi3.5020
• LO CHK. 2023. What Is the Impact of ChatGPT on Education? A Rapid Review of the Literature” Educ Sci 13(4): 410. https://doi.org/10.3390/educsci13040410.
  » https://doi.org/10.3390/educsci13040410
• MARTÍN-M A, ORDUNA-ME, THELWALL M & DELGADO LCE. 2018. Google Scholar, Web of Science, and Scopus: A systematic comparison of citations in 252 subject categories. J Informet 12(4): 1160-1177. https://doi.org/10.1016/j.joi.2018.09.002.
  » https://doi.org/10.1016/j.joi.2018.09.002
• MONGEON P & PAUL-HUS A. 2016. The coverage of Web of Science and Scopus journals: a comparative analysis. Scientometrics 106: 213-228. https://doi.org/10.1007/s11192-015-1765-5.
  » https://doi.org/10.1007/s11192-015-1765-5
• MORALES M. 2023 Learning in times of ChatGPT: The student and the teacher. Galileo University. Available in https://www.galileo.edu/ivn/noticias/aprendizaje-en-tiempos-de-chatgpt-el-estudiante-y-el-docente/
  » https://www.galileo.edu/ivn/noticias/aprendizaje-en-tiempos-de-chatgpt-el-estudiante-y-el-docente/
• NAIDU K & SEVNARAYAN K. 2023. ChatGPT: a growing invasion of artificial intelligence in online assessment in distance education. Online J Media Comm Technol 13(3): e202336. https://doi.org/10.30935/ojcmt/13291.
  » https://doi.org/10.30935/ojcmt/13291
• NIKOLIC S, SCOTT D, REZWANUL H, BELKINA M, GHULAM MH, GRUNDY S, LYDEN S, NEAL P & CAZ S. 2023. ChatGPT versus assessment of engineering education: a multidisciplinary and multi-institutional benchmarking and analysis of this artificial intelligence tool to investigate assessment integrity. Eur J Eng Educ 48(4): 559-614. https://doi-org.ezproxy-cicco.conacyt.gov.py/10.1080/03043797.2023.2213169.
• NOVO A, NOBRE A, SIMÃO J & PEREIRA P. 2020. Plagiarism and academic integrity in the information society. Online. Lisbon: Universidade Aberta, 97 p. eUAb Science Culture 10. https://doi.org/10.34627/ftpm-hq41
  » https://doi.org/10.34627/ftpm-hq41
• OPARA ECH, ADALIKWU ME T & TOLORUNLEKE CA. 2023. ChatGPT for Teaching, Learning and Research: Prospects and Challenges. Glob Acad J Humanit Soc Sci 5(2): 33-40. https://ssrn.com/abstract=4375470.
• PÉREZ EA. 2017. WoS and Scopus. The great allies of the researcher. https://doi.org/10.3916/escuela-de-autores-031
  » https://doi.org/10.3916/escuela-de-autores-031
• PESSANHA LPM, SANTOS ICD, HENRIQUES MV, CANÇADO R, VINAGRE RF & CASTRO ADC. 2023. Brazilian research trends in Ergology. Ciência & Saúde Coletiva 28: 521-536. https://doi.org/10.1590/1413-81232023282.09412022.
  » https://doi.org/10.1590/1413-81232023282.09412022
• RAHMAN M, TERANO HJR, RAHMAN N, SALAMZADEH A & RAHAMAN S. 2023. ChatGPT and Academic Research: A Review and Recommendations Based on Practical Examples. J Educ Manag Develop Stud 3(1): 1-12. doi: 10.52631/jemds.v3i1.175.
• RODE S DE M & LOPES SE. 2022. Ethics and integrity in scientific publishing. Open scientific communication. Collection PPGCI 50 years. Available at http://repositorio.ibict.br/bitstream/123456789/1223/1/PrincipeRode_ComunicacaoCien tificaAberta_2022.pdf
  » http://repositorio.ibict.br/bitstream/123456789/1223/1/PrincipeRode_ComunicacaoCien tificaAberta_2022.pdf
• ROUHIAINEN L. 2018. Artificial intelligence. Madrid: Editorial Encourages. Available in https://static0planetadelibroscom.cdnstatics.com/libros_contenido_extra/40/39308_Intel igencia_artificial.pdf
  » https://static0planetadelibroscom.cdnstatics.com/libros_contenido_extra/40/39308_Intel igencia_artificial.pdf
• SEVILLA UNIVERSITY LIBRARY. 2023. Artificial intelligence in research and university teaching. Available in https://bib.us.es/sites/bib3.us.es/files/investiga41.pdf
  » https://bib.us.es/sites/bib3.us.es/files/investiga41.pdf
• SOK S & HENG. 2023. ChatGPT for Education and Research: A Review of Benefits and Risks. http://dx.doi.org/10.2139/ssrn.4378735
  » http://dx.doi.org/10.2139/ssrn.4378735
• STOJANOV A. 2023. Learning with ChatGPT 3.5 as a more informed other: an autoethnographic study. Int J Educ Technol High Educ 20: 35. https://doi.org/10.1186/s41239-023-00404-7.
  » https://doi.org/10.1186/s41239-023-00404-7
• STRZELECKI A. 2023. To use or not to use ChatGPT in higher education? A Study on Student Acceptance and Use of Technology, Interactive Learning Environments, https://doi-org.ezproxy-cicco.conacyt.gov.py/10.1080/10494820.2023.2209881
  » https://doi-org.ezproxy-cicco.conacyt.gov.py/10.1080/10494820.2023.2209881
• TEIXEIRA CNG ET AL. 2023. The use of dental services in the last year in the Brazilian population: systematic review with meta-analysis. Sci Collect Health 28: 1087-1100. https://doi.org/10.1590/1413-81232023284.11452022.
  » https://doi.org/10.1590/1413-81232023284.11452022
• UNESCO. 2023a. Women and Men in Science still have more results, says the UN. https://news.un.org/pt/story/2023/02/1809607
  » https://news.un.org/pt/story/2023/02/1809607
• UNESCO. 2023b. ChatGPT and Artificial Intelligence in higher education. Quick Start Guide. https://www.iesalc.unesco.org/wp-content/uploads/2023/04/ChatGPT-e-Inteligencia-Artificial-en-la-educacio%CC%81n-superior-Gui%CC%81a-de- start-ra%CC%81pido_FINAL_ENG.pdf
  » https://www.iesalc.unesco.org/wp-content/uploads/2023/04/ChatGPT-e-Inteligencia-Artificial-en-la-educacio%CC%81n-superior-Gui%CC%81a-de- start-ra%CC%81pido_FINAL_ENG.pdf
• VAN LEEUWEN TN ET AL. 2001. Language biases in the coverage of the Science Citation Index and its consequences for international comparisons of national research performance. Scientometrics 51: 335-346. https://doi.org/10.1023/A:1010549719484.
  » https://doi.org/10.1023/A:1010549719484
• YILMAZ R & KARAOGLAN FG. 2023. The effect of generative artificial intelligence (AI)-based tool use on students’ computational thinking skills, programming self-efficacy and motivation. Computers and Education: Artificial Intelligence 4: 100147. https://www.scopus.com/inward/record.uri?eid=2-s2.085161306434&doi=10.1016%2fj.caeai.2023.100147&partnerID=40&md5=c2c84f4 a14753cf9e2776c2d600ea2d0.
• YOGESH KD ET AL. 2023.Opinion Paper: “So what if ChatGPT wrote it?” Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy, Int J Inf Manag 71: 102642. https://doi.org/10.1016/j.ijinfomgt.2023.102642.
  » https://doi.org/10.1016/j.ijinfomgt.2023.102642

Publication Dates

• Publication in this collection
  10 June 2024
• Date of issue
  2024

History

• Received
  08 Aug 2023
• Accepted
  20 Oct 2023