Accessibility / Report Error

A PhD dissertation in education written by artificial intelligence?

ABSTRACT

This paper analyses data and perceptions of graduate students in Education on the partial production of an article, a master's thesis and a PhD dissertation written by artificial intelligence (AI). The motivation for the study occurs due to the rise of AI for text generation, examining the trends and implications of algorithmic writing usage by graduate students. We analyzed data originating from 123 questionnaires, and primarily, the contents of 17 interviews conducted in 2021. Some conclusions were that 88.6% of the graduate students are unfamiliar with AI use for text production, though the aggregated frequency demonstrates a tendency to use it by 84.5% and the interviews, carried out in safeguarded conditions, by 70.6%; the admission of use varied among categories in the individual and in context scopes; "AI and human hybrid" and "standard" type texts emerge from content analysis; a notion of plagiarism by AI appears; moral and ethical issues come up both as conditions and as an opposition to the use of AI in written production.

Keywords:
Artificial Intelligence; Written Text; Education; Post-Graduation

RESUMO

O artigo analisa dados e percepções de mestrandos e doutorandos em Educação sobre a produção parcial de artigos, dissertações e teses de doutorado por inteligência artificial (IA). A motivação do estudo ocorre em razão do incremento da IA para a geração de textos, examinando as tendências e implicações de uso da escrita algorítmica por pós-graduandos. Foram examinados dados de 123 questionários e, principalmente, a análise de conteúdo de 17 entrevistas realizadas em 2021. Algumas conclusões: dos pós-graduandos, 88,6% desconhecem a IA para a produção de texto, mas a frequência agregada de questionário aponta tendência de uso em 84,5% e, em entrevista, em condições com ressalvas, em 70,6%; a admissão do uso varia por categorias nos âmbitos individual e de contexto; textos do tipo "híbrido: IA e humano" e "padrão" emergem da análise de conteúdo; configura-se uma noção de plágio por IA; a moral e a ética surgem tanto como condicionalidade quanto como oposição para o uso da IA na produção escrita.

Palavras-chave:
Inteligência Artificial; Texto Escrito; Educação; Pós-Graduação

RESUMEN

El artículo analiza datos y percepciones de estudiantes de máster y doctorado en Educación sobre la producción parcial de artículos, disertaciones y tesis doctorales por inteligencia artificial (IA). La motivación del estudio ocurre debido al incremento de la IA para la generación de textos, examinando las tendencias e implicaciones del uso de la escritura algorítmica por parte de los postgraduados. Se examinaron los datos de 123 cuestionarios y, principalmente, el análisis de contenido de 17 entrevistas efectuadas en 2021. Algunas conclusiones: el 88,6% de los posgraduados desconoce la IA para la producción de textos, pero la frecuencia agregada del cuestionario señala una tendencia de uso en el 84,5% y, en la entrevista, en condiciones con reservas, el 70,6%; la admisibilidad de uso varía por categorías en los ámbitos individual y de contexto; los textos del tipo "híbrido: IA y humano" y "estándar", emergen del análisis de contenido; se configura una noción de plagio por parte de la IA; la moral y la ética emergen tanto como condicionalidad como oposición para el uso de la IA en la producción escrita.

Palabras clave:
Inteligencia Artificial; Texto Escrito; Educación; Posgraduación

INTRODUCTION

"You are in my committee. You see the perfect piece of writing and you know it wasn't me [who wrote it], I find it a bit strange" [added emphasis]. (Moon, 26, Education, Ma).1 1 Fictitious name, followed by age, major and school level (Ma — Academic Master's; Mp — Professional Master's and Doctorate — PhD).

"First, I believe the title of master or doctor should go to the program, not to the citizen. … I think maybe we could create another category, because technology creates situations which we could not have imagined years ago" [added emphasis]. (Sun, 34, Education, Ma)

The quotes of the interviews performed with Moon and Sun, graduate students doing a Master's in Education, have led us to imagine the following scene: the defense of a dissertation2 2 The reference made in the title to a PhD dissertation is meant to represent a Graduate study written production, but in the paper's approach, we also include Master's theses and articles. in Education of which part was written by artificial intelligence (AI) and, taking part in the committee, not only humans, but an algorithm programmed to assess the writing and question the PhD student. In this particular scene, not only does the human participant have to defend the dissertation, the algorithm also acquires a "personality" against the other algorithm, the one which wrote part of the text. From this scene, we infer the notion that the writing algorithm turns the idea of authorship and author in the production of texts resulting from academic research into something more complex. The human part may merely function as a copyist associated or mixed with the academic text writing algorithm. The scene described here is simple, caricatural and imaginary in respect to the signaling of AI development and its applications. In the school system logic, AI has shown great development and several applications, mainly in what concerns writing assessment in teaching-learning processes and in educational experiences with the use of language models in text production. Outside of educational institutions, such as in the internet, it is possible to find text production algorithms' ads and, with greater innovation and improvement of these tools, there will certainly be discussions over the nature and ends of the knowledge produced in educational research with the use of AI. It is exactly this reflection which constitutes the focus of the present study. Feng and Law (2021)FENG, Shihui; LAW, Nancy. Mapping artificial in education research: a network based keyword analysis. International Journal of Artificial Intelligence in Education, v.31, n. 2, p. 277-303, 2021. https://dx.doi.org/10.1007/s40593-021-00244-4
https://dx.doi.org/10.1007/s40593-021-00...
mapped 1,830 articles about the knowledge produced by AI in research studies in Education. In doing so, they identified two major thematic fields: Intelligent Tutoring Systems (ITS) and the massive open online courses. Studies have addressed the automated assessment of texts, engaging behavior in studies, feedback generation, performance forecast and modeling, demonstrating associations between ITS and Natural Language Processing (NLP), among other aspects (Feng and Law, 2021FENG, Shihui; LAW, Nancy. Mapping artificial in education research: a network based keyword analysis. International Journal of Artificial Intelligence in Education, v.31, n. 2, p. 277-303, 2021. https://dx.doi.org/10.1007/s40593-021-00244-4
https://dx.doi.org/10.1007/s40593-021-00...
). NLP, as one of the facets of AI, deals with automatic generation and understanding of natural human languages (speech, writing, translation). Vicari's study (2018) on Educational technological AI-based trends, which covered prospect scenarios until year 2030, identified a tendency towards the general use of NLP products which included writing. Selwyn et al. (2020)SELWYN, Neil; HILLMAN, Thomas; EYNON, Rebecca; FERREIRA, Giselle; KNOX, Jeremy; MACGILCHRIST, Felicitas; SANCHO-GIL, Juana M. What's next for Ed-Tech? Critical hopes and concerns for the 2020s. Learning, Media and Technology, [S.l.], v. 45, n. 1, p. 1-6, 2020. https://doi.org/10.1080/17439884.2020.1694945
https://doi.org/10.1080/17439884.2020.16...
emphasized that, during the 2020s decade, with the advanced of AI in Education, a process of datafication will take place. In face of the many different models of technological language, "artificial intelligence will increasingly become the engine of education, and student data the fuel" (Selwyn et al., 2020SELWYN, Neil; HILLMAN, Thomas; EYNON, Rebecca; FERREIRA, Giselle; KNOX, Jeremy; MACGILCHRIST, Felicitas; SANCHO-GIL, Juana M. What's next for Ed-Tech? Critical hopes and concerns for the 2020s. Learning, Media and Technology, [S.l.], v. 45, n. 1, p. 1-6, 2020. https://doi.org/10.1080/17439884.2020.1694945
https://doi.org/10.1080/17439884.2020.16...
, p. 2).

Composition and written questions' assessment is one of the fields in which NPL and Machine Learning (ML) AI techniques have been used to evaluate written production in an automated manner based on criteria such as deviating from the topic and other items, proving itself as an alternative to minimize the human evaluator's effort and time use in identifying students' performance issues (Pinho et al., 2022PINHO, Cíntia Maria de Araújo; MOURA, Amanda Ferreira de; GASPAR, Marcos Antonio; NAPOLITANO, Domingos Márcio Rodrigues. Identificação de deficiências em textos educacionais com a aplicação de processamento de linguagem natural e aprendizado de máquina. ETD – Educação Temática Digital, [s.l.], v. 24, n. 2, p. 350-372, abr./jun. 2022. https://doi.org/10.20396/etd.v24i2.8660061
https://doi.org/10.20396/etd.v24i2.86600...
). From Vicari's (2018) and Feng's and Law's (2021) research, we may infer that there is a branch of AI in Education emerging from academic research combined or not with business companies, focusing on knowledge production and technological support for the teaching-learning process in the school system, in which written assessment is included. On the other hand, AI for writing in the school system, for instance, for the generation of an academic essay or article, is emerging and current in the years 2019 to 2023, in the scope of the creation and exploration of issues involved in using algorithms for text production.

When focusing on customized learning and tutoring, however, one NLP program has passed undetected: Transformer (Sharples, 2022SHARPLES, Mike. New AI tools that can write student essay require educators to rethink teaching and assessment. [S.l], 17 maio 2022. Disponível em: .https://blogs.lse.ac.uk/impactofsocialsciences/2022/05/17/new-ai-tools-that-can-write-student-essays-require-educators-to-rethink-teaching-and-assessment/. Acesso em: 27 jul.2022.
https://blogs.lse.ac.uk/impactofsocialsc...
). This AI technology, besides being useful in the learning process or as a creativity tool, may be used by students who wish to cheat (Sharples, 2022SHARPLES, Mike. New AI tools that can write student essay require educators to rethink teaching and assessment. [S.l], 17 maio 2022. Disponível em: .https://blogs.lse.ac.uk/impactofsocialsciences/2022/05/17/new-ai-tools-that-can-write-student-essays-require-educators-to-rethink-teaching-and-assessment/. Acesso em: 27 jul.2022.
https://blogs.lse.ac.uk/impactofsocialsc...
). Sharples (2022)SHARPLES, Mike. New AI tools that can write student essay require educators to rethink teaching and assessment. [S.l], 17 maio 2022. Disponível em: .https://blogs.lse.ac.uk/impactofsocialsciences/2022/05/17/new-ai-tools-that-can-write-student-essays-require-educators-to-rethink-teaching-and-assessment/. Acesso em: 27 jul.2022.
https://blogs.lse.ac.uk/impactofsocialsc...
, in regard to the GPT-3, one of the language models based on Transformer, highlights the possibility of generating a whole essay with a single user command, from the title to the references, in a way that the piece would not be classified as plagiarism, since the writing would have been generated, not copied. This AI-based language model foresees the next words, considering the previous ones in some written production. The human subject enters a topic in the system, and the tool presents samples, options for the user to continue writing in a coherent manner (Radford et al., 2019RADFORD, Alec; WU, Jeffrey; AMODEI, Dario; AMODEI, Daniela; CLARK, Jack; BRUNDAGE, Miles; SUTSKEVER, Ilya. Better language models and their implications [Original post]. [S.l.]: OpenAI, 14 fev. 2019. Disponível em: https://openai.com/blog/better-language-models/. Acesso em: 27 jul. 2022.
https://openai.com/blog/better-language-...
). Some AI essay writing records have shown problems such as the insertion of references which are pertinent to the topic, but actually nonexistent, fictional research, false allegations; plausible but non-existing quotes, and repetitive writing (Radford et al., 2019RADFORD, Alec; WU, Jeffrey; AMODEI, Dario; AMODEI, Daniela; CLARK, Jack; BRUNDAGE, Miles; SUTSKEVER, Ilya. Better language models and their implications [Original post]. [S.l.]: OpenAI, 14 fev. 2019. Disponível em: https://openai.com/blog/better-language-models/. Acesso em: 27 jul. 2022.
https://openai.com/blog/better-language-...
; Otsuki, 2020OTSUKI, Grant Jun. Ok computer: To prevent students cheating with AI text-generators, we should bring them into the classroom. [S.l.]: The Conversation, 23 jan. 2020. Disponível em: https://theconversation.com/ok-computer-to-prevent-students-cheating-with-ai-text-generators-we-should-bring-them-into-the-classroom-129905. Acesso em: 27 jul. 2022.
https://theconversation.com/ok-computer-...
; Fyfe, 2022FYFE, Paul. How to cheat on you final paper: Assigning AI for student writing. AI & Society: Knowledge, Culture and Communication, London, 10 mar. 2022. https://doi.org/10.1007/s00146-022-01397-z
https://doi.org/10.1007/s00146-022-01397...
; Sharples, 2022SHARPLES, Mike. New AI tools that can write student essay require educators to rethink teaching and assessment. [S.l], 17 maio 2022. Disponível em: .https://blogs.lse.ac.uk/impactofsocialsciences/2022/05/17/new-ai-tools-that-can-write-student-essays-require-educators-to-rethink-teaching-and-assessment/. Acesso em: 27 jul.2022.
https://blogs.lse.ac.uk/impactofsocialsc...
). Fyfe (2022)FYFE, Paul. How to cheat on you final paper: Assigning AI for student writing. AI & Society: Knowledge, Culture and Communication, London, 10 mar. 2022. https://doi.org/10.1007/s00146-022-01397-z
https://doi.org/10.1007/s00146-022-01397...
reports, on a paper, a learning experience using GPT-2 version, in which the content generated by the program integrated itself to the students' own reflections — with no AI use — in the writing of twenty essays developed as the course's final assignment. The author neither loses sight of the preliminary question about ethics and the use of AI in writing, nor of what would be deemed plagiarism in writing. He also keeps in mind the conditions, if any, that should be observed for algorithm use to support students' writing, besides the hypothesis that sustains that working with algorithms may change the way we think of writing or even of our authenticity and creativity (Fyfe, 2022FYFE, Paul. How to cheat on you final paper: Assigning AI for student writing. AI & Society: Knowledge, Culture and Communication, London, 10 mar. 2022. https://doi.org/10.1007/s00146-022-01397-z
https://doi.org/10.1007/s00146-022-01397...
). Fyfe (2022)FYFE, Paul. How to cheat on you final paper: Assigning AI for student writing. AI & Society: Knowledge, Culture and Communication, London, 10 mar. 2022. https://doi.org/10.1007/s00146-022-01397-z
https://doi.org/10.1007/s00146-022-01397...
states that some students criticized the AI generated texts mentioning they differed from their points of view and writing style; others were surprised, since the AI generated writing that was similar to theirs and even expressed their own thoughts.

Some students verified that part of the essay content flowed in consonance with their personal writing and that the AI language model adopted helped them articulate their developing ideas or those they had to make an effort to express, among other issues. In general, the course participants did not conclude one way or the other that writing through AI meant plagiarism, and the author affirms that the possibility of this practice becoming a fraud is a complex matter to study and explain (Fyfe, 2022FYFE, Paul. How to cheat on you final paper: Assigning AI for student writing. AI & Society: Knowledge, Culture and Communication, London, 10 mar. 2022. https://doi.org/10.1007/s00146-022-01397-z
https://doi.org/10.1007/s00146-022-01397...
). The writer ends the article stating that: "Embedding students within these debates helps show them not only the important issues at stake, but invites them into the evolving ethical project of dealing with AI in our world" (Fyfe, 2022FYFE, Paul. How to cheat on you final paper: Assigning AI for student writing. AI & Society: Knowledge, Culture and Communication, London, 10 mar. 2022. https://doi.org/10.1007/s00146-022-01397-z
https://doi.org/10.1007/s00146-022-01397...
, n.p.).

It becomes evident, in the author's experiment, the intention to bring to Academia the reflection on AI use for writing, and, mainly, based on its results, on the matter of if and how it would be possible to work ethically and productively with this type of technological language, enriching academic discussion in topics such as plagiarism, authorship and writing pedagogy (Fyfe, 2022FYFE, Paul. How to cheat on you final paper: Assigning AI for student writing. AI & Society: Knowledge, Culture and Communication, London, 10 mar. 2022. https://doi.org/10.1007/s00146-022-01397-z
https://doi.org/10.1007/s00146-022-01397...
). Nevertheless, moving in a different direction from research in the academic world, there are business companies and individuals interested in profiting that, at times, register algorithm patents not necessarily aiming at teaching and learning writing. These agents invest in the consumer market of text production apps, replacing the work of the author with a writing algorithm. It is in this second trend, via internet, that AI writing production tools, outside of the school system teaching-learning logic, tend to show a higher increase of and incremental use by different subjects for fraud or otherwise. While, on one hand, as Pinho et al. (2022)PINHO, Cíntia Maria de Araújo; MOURA, Amanda Ferreira de; GASPAR, Marcos Antonio; NAPOLITANO, Domingos Márcio Rodrigues. Identificação de deficiências em textos educacionais com a aplicação de processamento de linguagem natural e aprendizado de máquina. ETD – Educação Temática Digital, [s.l.], v. 24, n. 2, p. 350-372, abr./jun. 2022. https://doi.org/10.20396/etd.v24i2.8660061
https://doi.org/10.20396/etd.v24i2.86600...
argue that the use of AI could contribute to reduce time and effort to assess students' written work, on the other hand, the same tools appear in apps' advertisement proposing to save users' time and effort for writing texts by paraphrasing and summarizing entire texts, allegedly with no plagiarism. We bring this up here only to make it clear that, regardless of the school system principles and goals for AI use, we find ads on social networks of algorithms which produce written summaries. On Instagram, for instance, there are several posts advertising how to "summarize any text in 1 minute" and "rewrite a text without plagiarizing it". With regard to the highly complex issue of students' autonomy and authorship in the face of AI advancements, the existence of algorithms which rewrite articles changing sentences and synonyms, both in English and in Portuguese (Spinbot, Plagiarism, Spinner, Cleaver Spinne, Word Spinner), provoke reflections about originality, plagiarism, and the impacts on knowledge production (Sayuri, 2019SAYURI, Juliana. Aplicativos que reescrevem artigos acendem alerta na acadêmica. Folha de São Paulo, São Paulo, ano 1-2, n. 34.130, 7 abr. 2019. Ilustríssima, 3.). Furthermore, there are also algorithms able to mimic any writer's style, for instance, Miguel de Cervantes's. News on ML and NLP Technologies have been published with the purpose of affirming that: "If a machine can learn an author's style, it can also capture many other details of the language, and this is of great use for any kind of practical solutions to work with the written text [added emphasis]" (Barbero, 2018BARBERO, Álvaro. Escritura a la carta -Inteligencia artificial y literatura: la máquina que escribe. [Entrevista cedida ao] Canal 24h da TVE "Zoom Net". Reprodução da entrevista no sítio eletrônico do Instituto de Enginiería del Conocimiento, [s.l.], 8 fev. 2018. Disponível em: https://www.iic.uam.es/noticias/inteligencia-artificial-literatura-la-maquina-escribe. Acesso em: 27 jul. 2022.
https://www.iic.uam.es/noticias/intelige...
, n.p.).

All these elements represent the "tip of the iceberg". They are pieces of a "puzzle" in the configuration of algorithms to write academic texts using AI resources, such as the partial writing of a PhD dissertation, a master's thesis or an article.

Sun's thoughts exposed in the opening of this article are aligned with Araújo's (2016), when he affirms: "Thus, a PhD dissertation totally generated by algorithms would not allow us to recognize a researcher as deserving of the title of ‘doctor’ in a particular field of scientific knowledge" [added emphasis] (p. 96). Two aspects call our attention in Araújo's (2016) thoughts: first is the fact that he includes the adverb "totally" in his statement regarding the writing of a dissertation by an algorithm and, second, that he refers to a "particular" field of knowledge. The question becomes unavoidable: Would it then be admissible for the algorithm to write "part" of a PhD dissertation in a field of knowledge such as Education? Araújo (2016)ARAÚJO, Marcelo. O uso de inteligência artificial para a geração automatizada de textos acadêmicos: Plágio ou meta-autoria? Logeion: Filosofia da Informação, v. 3, n. 1, p. 89-107, 2016. https://doi.org/10.21728/logeion.2016v3n1.p89-107
https://doi.org/10.21728/logeion.2016v3n...
, himself, while reviewing literature about AI text production, mentions several situations using this tool, from journalistic articles to books, and refers to the development of an algorithm to write PhD dissertations. Regarding the latest aspect, in consultation with the original source cited by Araújo (2016)ARAÚJO, Marcelo. O uso de inteligência artificial para a geração automatizada de textos acadêmicos: Plágio ou meta-autoria? Logeion: Filosofia da Informação, v. 3, n. 1, p. 89-107, 2016. https://doi.org/10.21728/logeion.2016v3n1.p89-107
https://doi.org/10.21728/logeion.2016v3n...
, we read the interview "Philip Parker's Trick for Authoring Over 1 million books: Don't Write", in which the author cited in the title talks about a project to design an algorithm to write a complete dissertation, avoiding the "pains" of a four year PhD course. According to Araújo (2016)ARAÚJO, Marcelo. O uso de inteligência artificial para a geração automatizada de textos acadêmicos: Plágio ou meta-autoria? Logeion: Filosofia da Informação, v. 3, n. 1, p. 89-107, 2016. https://doi.org/10.21728/logeion.2016v3n1.p89-107
https://doi.org/10.21728/logeion.2016v3n...
, a dissertation generated by an algorithm subverts the issue of originality. For the author, such work could not by disqualified as plagiarism, since the algorithm could be programmed to manage text transcription, including in its formatting the number of words and bibliographical notes presented on the written piece. To the author, in the future, plagiarism will be a minor problem, given that it may be battled by means of software that detects such practice in written production. Nevertheless, it is worth noticing that the same technological sophistication of applications that write paraphrases and summaries in lieu of a subject's intellectual work tends to be accompanied by major innovations to avoid pieces of software that verify similarities between texts and identify intentional frauds by users, among them plagiarism. This way, in such a scenario, we cannot ignore that frauds in academic work (Sureda-Negre et al., 2007), until now artisanal, have begun to reach a larger industrial scope and influence the quality and credibility of higher education (Comas Forgas et al., 2021COMAS FORGAS, Rubén.; CERDÀ NAVARRO, Antoni.; OLIVER TROBAT, Miquel.; SUREDA NEGRE, Jaume. Contract cheatin in Spain: identificación and analysis of academic "ghost-writing". In: GÓMEZ GARCÍA, Geraldo.; NAVAS-PAREJO, Magdalena Ramos.; RODRÍGUEZ JIMÉNEZ, Carmen.; DE LA CRUZ CAMPOS, Juan Carlos (Eds). Teoría y práctica en investigación educativa: una perspectiva internacional. Madrid: Dykyson, S.L, 2021. p. 473-485. Disponível em: https://iapost.uib.es/a/f31e9a9c514ae99e127c1e0982b5e860f20ef481/9788413771748_libro_dykinson_capitulo.pdf. Acesso em: 27 jul. 2022.
https://iapost.uib.es/a/f31e9a9c514ae99e...
). Still in terms of scenarios, we bring to the debate the thoughts of Barakina et al. (2021)BARAKINA, Elena Y.; POPOVA, Anna V.; GOROKHOVA, Svetlana; VOSKOVSKAYA, Angela S. Digital technologies and artificial intelligence Technologies in education. European Journal of Contemporary Education, v. 10, n. 2, p. 285-296, 2021. https://doi.org/10.13187/ejced.2021.2.285
https://doi.org/10.13187/ejced.2021.2.28...
, who highlight the ways to develop the digital technologies and AI for Education, in the context of the processes of learning support. These authors show a concern with the formation of the subject when approaching scientific research and practices using AI, since the use of these technologies should not revolve only around the achievement of objectives of a program or the development of skills, but should also consider the component of the formation of the subject's personality, as an individual belonging to a society.

Therefore, it is important to highlight that the algorithm which writes texts does not work as an invisible structure that acts outside the subject, because there is another party in this relationship, a human being, that acts and interferes in a certain way in the tool itself (Lopes, 2020LOPES, Carlos. O plágio como questão pública. In: NOVO, Ana; NOBEL, Ana; SIMÃO, João; PEREIRA, Pedro (orgs.). Plágio e integridade acadêmica na sociedade da informação. v. 10. Lisboa: UAB Portugal, 2020. p. 15-33. https://doi.org/10.34627/ftpm-hq41
https://doi.org/10.34627/ftpm-hq41...
).

The study herein presented, in what concerns this paper's data, took place at the historical moment when GPT-2 and GPT-3 were used for AI text writing and is unique in its approach of real subjects, such as master's and doctorate students in the field of Education of a Brazilian public university — revealing, by means of a qualitative and quantitative study, their perceptions regarding the trends and implications of AI use in part of their research studies. This study also contributes with a comprehensive mapping of categories intending to provoke new thoughts, research and institutional interventions regarding the subject matter.

OBJECTIVE AND RESEARCH QUESTIONS

The objective of this paper is to present and analyze the perception of Master's and PhD students in the field of Education about the partial production of articles, theses and dissertations by AI in research in Education. Starting from data on and perceptions by Master's and PhD students, we established a correlation between the level of the course and the graduate students' opinions about the admissibility of the use of AI in written production. We also attained a categorization by content analysis.

The research questions of the current study are the following:

  • Do Master's and PhD students in the field of Education know AI for writing texts?

  • Both by quantitative and qualitative frequencies, by category, how is the tendency to use AI to create, organize and write academic texts distributed amongst Master's and PhD students in the field of Education?

  • Is there a significant difference in the way Master's and PhD students position themselves concerning the use of AI for the creation, organization and writing3 3 When adequate, AI-COW will correspond to the acronym used in the paper for Artificial Intelligence – Creation, Organization and Writing. of academic texts?

  • Which are the emerging categories in the positioning of graduate students towards admitting or not the partial use of AI in academic writing and what do the data indicate?

PARTICIPANTS

The participants of the research are Brazilian students of two Graduate Programs on the field of Education in the School of Education (FE) at the University of Brasília (UnB). One Graduate Program had students from the Professional Master's Program,4 4 The Academic Master's Degree has the purpose of preparing the professionals for teaching and research, while the Professional Master's degree is targeted to capacity building, considering the study of techniques, processes, and topics responding to the demands of the job market. https://www.gov.br/capes/pt-br/acesso-a-informacao/perguntas-frequentes/sobre-a-cap. Access on: July 27, 2022. and the other had students from the Academic Master's and PhD Programs. In 2021, the population of enrolled students amounted to 341 students, among them 133 graduates of the Academic Master's, 118 from the Professional Master's and 90 from the PhD programs. We obtained a sample of 123 questionnaire respondents, representing 36% of the students' population. From the 123 questionnaire respondents, 44 (35,8%) were from the Professional Master's, 41 (33,3%) from the Academic Master's and 38 (30,9%) from the PhD level. Regarding sex/gender: 75,6% were women, and 24,4% were men. The higher number of women is correlated both to a significant group of graduate students who are in the field of Education (Pedagogy) and to the graduate course itself, which is in Education. Regarding professional experience, 70% of the respondents were professors or teachers whose work spans early childhood education to undergraduate and graduate levels. We interviewed 17 graduate students: four PhD students and 13 Master's students, six from the Professional Master's and seven from the Academic Master's. Overall, six men and 11 women were interviewed.

METHOD

The study has a descriptive and exploratory nature, correlating both quantitative and qualitative data, mainly the latter. The survey was performed in 2021. All the graduate students agreed, by signing a Free and Informed Consent Term, to participate in the research. Forty six of the 123 questionnaire respondents were willing to participate in the interviews, informing their contact information in the appropriate space in the questionnaire's form, but only 17 graduates were interviewed. The limit of 17 interviewees in the convenience sample, in relation to the 46 available to the interview, was due to a combination of factors: a. some students did not provide their proper contact information to participate in the interviews; b. some did not accept to participate in the study, due to their priority to conclude their theses or dissertations; c. effects of the Covid-19,5 5 Covid-19 is a disease caused by the coronavirus, which generated a pandemic of global scale. In Brazil, the first case was registered in February 2020, and the declaration of community transmission in the country took place in March of the same year (cf. Agência Brasil, 2021). such as illness, death of family members, intensification of virtual classes and workload of graduate students, led to a certain exhaustion of the respondents to demands of the online interviews, and even to answer questionnaires; among other reasons.

The research questionnaire, apart from profile data (sex, age, undergraduate studies, level of the course), elicited answers to two questions: a. if the graduates knew AI for writing texts; b. if they would be willing to use AI to create, write and organize an academic text, with options in a frequency scale.6 6 Given the extension of the data from the questionnaire and the interviews, which included other research topics, in this paper we restricted ourselves to only two of the main questions directly related to the use of AI in text production. The results of the first question were presented to the respondents as a dichotomous option: "no" or "yes". For the second question, the options were given in a frequency scale of five categories: "never", "rarely", "occasionally", "frequently" and "very frequently". The statistical differences between the Professional Master's students and the Academic Master's and PhD students were verified by using contingency tables (cross-tabs), in which χ2 statistics were also applied to identify the residues typified in the categories.

In the interviews stage, questions were made which were aligned to those of the questionnaire, added by a hypothetical question about admitting or not the partial use of AI in the production of written material, in reviewing academic literature or in methodology. In the interviews stage, we used the thematic-categorical content analysis (Bardin, 2011BARDIN, Laurence. Análise de conteúdo. São Paulo: Edições 70, 2011.), fixating the register units, the context units and the numerical indexes. We applied the following stages in the analysis process: fluctuating reading of the interviews; content exploration; information treatment; and inference realization (Bardin, 2011BARDIN, Laurence. Análise de conteúdo. São Paulo: Edições 70, 2011.).

RESULTS AND DISCUSSION

CONCERNING KNOWLEDGE AND USE OF ARTIFICIAL INTELLIGENCE BY GRADUATE STUDENTS IN THE PRODUCTION OF THEIR TEXTS

In Table 17 7 The values in the same line which do not share the same subindex (a, b) are significantly different with p<0.05. The percentages in bold indicate cells with residues superior to 1.96 (positive association), and the percentages in italics indicate cells with residues inferior to -1.96 (negative association). below we present data concerning the answers to two of the questions in the questionnaire, including their distribution by course level:

Table 1
Knowledge of artificial intelligence by graduate students and frequency of use in their academic texts.

From what is shown in Question 1 (Table 1), 88.6% of graduates were unfamiliar with AI for writing, whereas 11.4% had knowledge about it. We did not find significant statistical differences between the groups (Professional Master's, Academic Master's and PhD) (p=0,434). From the interviews, the following result was obtained: 83.25% did not know about AI for writing and 17.65% showed knowledge about such technological resources. Therefore, there is an alignment between the data from the questionnaire and the interviews regarding the unfamiliarity of graduate students about the AI tool to generate written texts. It is worth highlighting that one male PhD and two female Master's students, amongst 17 interviewees, declared to know AI to produce text. Nevertheless, the notion of familiarity was restricted to a generic sense of the AI functionality, without representing learning or usage experience by the graduates. This notion of familiarity by one of the female Master's students interviewed was accompanied by a declaration of disbelief as she accessed web pages that hosted AI for text production:

I was even impressed… an application where you put a word and it already gives you a ready paragraph… I can hardly believe … it was a look of astonishment, really. …it shows hundreds of options, of possibilities, of applications that offer you, right there, ways for you to write a text in a quicker form, and the advertisement says: "Why waste time thinking about the construction of one text if you can, in a more practical way [added emphasis], write [using the tool]?". (Bellatrix, 32, Education, Mp)

The level of knowledge — or even lack of knowledge — on the subject of AI for the production of an academic text is followed by feelings of fear, associated with the idea of danger and/or of surprise, linking it to something unexpected and just found. Words such as "speed", "economy", "efficacy", "time", among others, are included in the context of the web pages that advertise the use of AI in the production of a text. The content of these pages, with versions in 11 foreign languages, informs that the AI object receives the user's instruction for writing, and their disseminators announce that the texts produced are of high quality and do not leave traces of plagiarism (Rewrite Guru, s.d.). This is only one type of algorithm that has specific properties to rewrite papers. From our research data, the number of graduate students who are familiar with such tools in that level of elaboration is minimal. This turns the issue into something less worrying and relevant in terms of discussions in the educational and research institutions. at the time of the interviews, none of the three graduates who knew AI for text production answered that they would use such a tool to write.

Concerning the questionnaires' answer about how often they would use an AI capable of aiding them in creating, organizing and writing academic texts (Question 2, Table 1), the category with the highest percentage was "occasionally", with 37.4%, followed by "frequently", with 21.1%, "rarely", with 17.9%, "never", with 15.4%, and "most frequently", with 8.1%. In general, the data indicated that there is a previous disposition of graduates for the use of AI for writing. There are no significant differences between groups (p=0,009). It is worth mentioning what the residues signal: a. in the category "frequently", in the Professional Master's group, there is a positive association (27.3% compared with an average of 21.1%), and the percentage of PhD students is lower (7.9% compared with 21.1%); b. in the category "rarely", and in the group of Professional Master's, there is a positive association (27.3% compared with 17.9%), and a lower percentage is found in the group of Academic Master's (2.4% compared with an average of 17.9%); c. in the category "occasionally", the percentage of Professional Master's students is lower in relation to the average (20.5% compared with 37.4%). Even observing a percentage dispersion between the categories, the tendency of the use of AI-COW is evident.

In the interviews stage, the same question, not regarding the frequency, but rather the use or not of AI in texts, obtained the percentage of use of 17.65%. Among the interviewees, only one female Academic Master's student chose to not answer, saying: "I do not know. I do not know how to answer" (Electra, 27, Ma). The other graduates answered that they would not use AI-COW in their texts. The discrepancy between the questionnaire data (84.5%), indicating the tendency of using AI in writing, and that of the interviews (17.65%) is related to the fact that the questionnaire is answered anonymously, and the sample of interviewees was a convenience one. That way, it was not possible to identify previously, between the interviewees, who would claim to use AI to write an academic text in the questionnaire. However, to unveil tendential aspects in relation to the use of AI in the process of writing, we included in the interview script a hypothetical question for the graduates to position themselves about the admissibility of using the tool, as shown in the next section.

ABOUT THE ADMISSIBILITY OF ARTIFICIAL INTELLIGENCE USE IN AN ACADEMIC TEXT: THE OPINIONS OF GRADUATE STUDENTS AND EMERGING CATEGORIES IN THE INTERVIEWS

The following hypothetic question was made to the interviewed students: do you agree or disagree that it is admissible to use artificial intelligence to write paragraphs or sentences when writing the contents of a thesis or a PhD dissertation, specifically when reviewing the literature about what you have already written or even about the research methodology, without it being a problem to scientific production? Justify your answer.

We did not identify any alignment to "admissibility" in categorical terms, but we found answers aligned with a "conditioned alignment", which involves a predisposition to use AI under certain conditions. "Non-admissibility", however, was identified under a viewpoint which, while defending some principle, approach, or consequence (among others), disagreed on the use of AI for the production of academic writing in graduate studies under any condition or situation.

As highlighted, only three (17.65%) of the 17 interviewed admitted the use of AI in their own texts. Nevertheless, when we changed the question to a hypothetic formulation regarding the admissibility of AI for a literature review and research methodology or in the exploration of categories or subcategories, this number increased to 12 (70.6%) in the classification associated with conditioned admissibility. By level of education, five graduate students from the Professional Master's program, five from the Academic Master's and two PhD students declared to be favorable to admissibility with reservations.8 8 Admissibility with reservations in this paper bears the same meaning as conditioned admissibility. The others, one from the Professional Master's, two from the Academic Master's and two PhD students were against AI use.

We may infer that the change in the initial figure of 17.65% of affirmative answers regarding the use of AI in the graduate students' own texts to a 70.6% figure in the interview data, which later approximated to 84.5% alleging a tendency to use in the questionnaire, was the result of the effect of the hypothetical question. The hypothetical type of question has a potential to reveal a tendency of use, since it concerns an imaginary situation, differing from questions of an evaluating nature in regard to the attitude of the subject facing a certain situation which involves moral and ethical issues of the production of scientific knowledge, such as articles, theses and PhD dissertations. In sum, in the reported case, the hypothetical question created a distance for the graduate student to deal with his or her practice in using AI, facing something unknown in terms of knowledge and effective appropriation, to approach a situation of possible admissibility with reservations in the future.

From the hypothetical question's answers, we created categories and subcategories. They were defined after a fluctuating reading of the content of the interviews (Bardin, 2011BARDIN, Laurence. Análise de conteúdo. São Paulo: Edições 70, 2011.) and their systematic review.

It is important to present the set of emerging categories and subcategories in order to, then, select, expose, and examine the distribution of some of them through the positioning of the graduate students, such as "conditioned admissibility" and "non-admissibility". The numbers on Chart 1 refers to the number of categories and subcategories which appeared only once in each interview.

Chart 1
Categories and subcategories in the content of interviews about Artificial Intelligence – Creation, Organization and Writing use.

As Chart 1 shows, the emerging categories and subcategories are not restrained in "methods" and "literature review", but have been extended to other items. Categories and subcategories are understood from the classification of the position of graduate students and from the respective context unit.

Due to the amount of research data and information, we will deal in this paper only with the direct relation between the "types of texts" and the categories presented as "conditioned admissibility" for the use of AI-COW.

"TYPES OF TEXT", CONCEPT AND CONDITIONED ADMISSIBILITY IN THE USE OF ARTIFICIAL INTELLIGENCE – CREATION, ORGANIZATION AND WRITING

The "types of text" and "literature review" (Chart 1) have reached the greatest number of characterized and understandable direct references in context units about the use of AI-COW. The category "types of text", in association with the subcategories representing specific configurations of generating and formatting written production, show four text types: a. "author's referential text"; b. "hybrid text: AI and human"; c. "artisanal text"; and d. "standard text". In the graduate students' responses, we found co-occurrences9 9 The cooccurrences take place due to the association between two or more words or topics (Bardin, 2011) which were present in the content of the interviews and appeared concomitantly or in other parts of the material analyzed, at times contradicting one another. That is, the graduate student "defends" the "artisanal" text while criticizing the AI generated "standard" or "hybrid" texts: "AI" and "human" showing up in the same sentence or in an answer to a subsequent question. Such data on a table would not be intelligible in relation to the numeric datum by level of course (Master's and PhD) or text typology. when they defended the "artisanal text" and criticized the "hybrid" and "standard" texts generated by AI. In this paper, we present and discuss one of these text formats in cooccurrence, the "author's referential text". This type of text is derived from the "artisanal text", but guards a certain distinction not only regarding the latter, but also in relation to the "hybrid text: AI and human", which will be better described when we discuss the context units.

In Chart 2, we present the type of text and its respective concept in context units.

Chart 2
Text types and senses attributed in opposition or adherence relation to Artificial Intelligence – Creation, Organization and Writing.

The frequency of the category "types of text", with 12 enumerations (Chart 1), shows how important reporting one's ideas in writing is as a marker of social distinction among graduate students. The content that follows clarifies the sense of social distinction and belonging connected with writing: "you are not capable of writing a text and you want to write a scientific text? First, you must develop that ability, and then you may join this researchers' club [added emphasis]" (Sol, 34, Education, Ma). Not less important was the "writing" category, mentioned ten times.

A total of three (17.65%)10 10 The two graduate students who answered thar they would use AI-COW in their own text are included in this figure. The third positioning was presented when answering a hypothetical question on admmitance or nonadmittance of AI-COW use. master students presented direct references to the "hybrid text: AI and human" subcategory, varying the use of terms such as "algorithm", "software", "program" and "machine" — sometimes associating them with human action and other categories conditioning use; and, at other moments, presenting the characteristic of AI itself for the educational process of the graduate student at the university. Among the PhD students, only one (5.88%) made a reference to the use of AI-COW and only in an article.

The complexity of the "hybrid text: AI and human" has correlations with other categories and subcategories concerning "writing", "literature review" and "ethic and collective morality", positioning the Master's students in "conditioned admissibility", as demonstrated in the following tables. It is worth mentioning that the quotes in the different categories belong to the same Master's student (Chart 3). The "writing" category maintains a direct relation with the form of textual writing, and it is present and evident in the quote by the interviewed student. The subjective expression to "unblock" the writing is connected with the feeling of "angst" represented in the notion of a "social time" in argumentative writing in which little or nothing flows, mobilizing the student to search for some tool to unblock the flow of ideas. This idea is also mentioned by Sharples (2022)SHARPLES, Mike. New AI tools that can write student essay require educators to rethink teaching and assessment. [S.l], 17 maio 2022. Disponível em: .https://blogs.lse.ac.uk/impactofsocialsciences/2022/05/17/new-ai-tools-that-can-write-student-essays-require-educators-to-rethink-teaching-and-assessment/. Acesso em: 27 jul.2022.
https://blogs.lse.ac.uk/impactofsocialsc...
and Fyfe (2022)FYFE, Paul. How to cheat on you final paper: Assigning AI for student writing. AI & Society: Knowledge, Culture and Communication, London, 10 mar. 2022. https://doi.org/10.1007/s00146-022-01397-z
https://doi.org/10.1007/s00146-022-01397...
. Fyfe's paper mentions a quote from a student emphasizing the use of AI for writing in the hybrid perspective in a less oppositional manner, looking at it more like a bridge connecting ideas: "AI assistants were not meant to replace or impersonate humans but provide a bridge that connects our ideas with theirs — a hybrid" (Fyfe, 2022FYFE, Paul. How to cheat on you final paper: Assigning AI for student writing. AI & Society: Knowledge, Culture and Communication, London, 10 mar. 2022. https://doi.org/10.1007/s00146-022-01397-z
https://doi.org/10.1007/s00146-022-01397...
, Results and discussion section). Master's students, many times, come from an educational background where there was no experience with scientific work or even article, essay or monograph writing, or they suffer from a socio-emotional writer's block. Thus, the student faces a demand for the production of written texts, which, among other criteria, depend on where the text will be published, as journals will also ask for relevant original and unprecedented contributions to their specific knowledge field, hence the fear of failure and exposure. This situation also occurs with PhD students. Therefore, for the student, AI-COW is a type of support, creating a sense of quality, trust and security, working as a virtual inductor of the unblocking of writing, which, beyond the practical and socio-emotional aspects of such unblocking, has moral and ethical implications.

Chart 3
Type of text in correlation with writing and ethic and collective morality for conditioned admissibility of Artificial Intelligence – Creation, Organization and Writing and context unit.

A Professional Master's student answered at one point that she would not use AI-COW in her own text; however, on a hypothetical level, in another content register, when discussing the admissibility of use, she declared herself favorable to "admissibility with reservations" (Chart 4).

Chart 4
Type of text in correlation with the literature review and ethics and collective morality for Artificial Intelligence – Creation, Organization and Writing conditioned admissibility and context unit categories.

The third positioning of a Master's student in the category "type of text", subcategory "hybrid text: AI and human", is by a graduate student from the academic course. His position regarding the hypothetic use of AI-COW focuses on AI functionalities and on graduate students' formative processes (teaching and learning) at the university, starting with certain characteristics of the tool to be used in writing, as follows:

scheduling, searching and finalizing mechanisms make use of artificial intelligence, so they are already used today to aid in research, to help build a theoretical basis… [university should] invest in artificial intelligence to assist in the production of academic texts…, but not its totality [added emphasis]. (Antares, 54, Computer Sciences, Ma)

This graduate student, coming from a Computer Sciences major and now involved in a teacher training in technology,11 11 Information about professional performance was extracted from the interviewee's profile. is aligned with the use of AI in the teaching-learning process, in analogy to the use of technological resources which search for book references in databases and format the references, but he claims, nevertheless, AI would not be used to organize the structure and perform the "total" writing of a text, replacing the human writer. To summarize, in his viewpoint the institutionalized use of AI in writing is part of the process of academic schooling, similar to the approach used in studies and research already performed in AI and Education, with registered tendencies and experiences (Vicari, 2018VICARI, Rosa Maria. Tendências em inteligência artificial na educação no período de 2017-2030: sumário executivo. Brasília: SENAI, SESI, 2018.; Feng and Law, 2021FENG, Shihui; LAW, Nancy. Mapping artificial in education research: a network based keyword analysis. International Journal of Artificial Intelligence in Education, v.31, n. 2, p. 277-303, 2021. https://dx.doi.org/10.1007/s40593-021-00244-4
https://dx.doi.org/10.1007/s40593-021-00...
; Fyfe, 2022FYFE, Paul. How to cheat on you final paper: Assigning AI for student writing. AI & Society: Knowledge, Culture and Communication, London, 10 mar. 2022. https://doi.org/10.1007/s00146-022-01397-z
https://doi.org/10.1007/s00146-022-01397...
).

Finally, there is the register of the only PhD student who, in the interview, affirmed she would use AI-COW to write an article, having something of her own authorship as the original source, showing a posture clearly aligned to the "non-admissibility" for thesis and dissertation writing. Thus, her writing production is identified as "artisanal text" in what concerns the origin, with no AI-COW interference during the search for sources of a different authorship other than her own. Regarding technical criteria, text production would be hybrid due to the use of the algorithm, which would work on her own previous text, generating a second written production. This procedure shows fine borders due to its specificity, since there is a previous writing by the author generating a second piece of writing with the use of AI. However, the PhD student's position, which moves between types of text, allows us to include her in the general category of "conditioned admissibility" for the use of AI in text production.

In Chart 5, as follows, we present the "conditioned admissibility" categories e subcategories, directly associated to "type of text", but there is also an expansion to include other emerging, interconnected categories. These categories and subcategories do not refer directly to contents associated with some type of text, but represent conditions for the use of AI-COW. For instance, the "ethic and collective morality" category and the "collective construction" subcategory are included in the context of "admissibility with reservations" as to the use of AI-COW in academic writing such as articles, thesis and PhD dissertation. In the same table, it is possible to identify mentions, in categories and subcategories, to "non-admissibility of AI-COW". This suggests some points of tension and conflict in respect to discussion and decision on AI use for academic research (article, thesis and PhD dissertation) in Graduate Studies' programs, such as the one in Education. Next, we present a selection and highlights of some admissibility and non-admissibility categories and subcategories. The use of AI-COW in the "literature review" obtained ten subcategories derived from "non-admissibility", and two from "conditioned admissibility". The first mention to "conditioned admissibility" emerges from the categories of "literature review" (subcategory "bibliographical search") and "ethics and collective morality" ("plagiarism punishment" subcategory), maintaining a direct relation in content with the "hybrid: AI and human" type of text, as shown in Chart 3. From a strictly informational "literature review", even with implications to the type of appropriation by the researcher in reading and understanding AI set contents, there is still a distance to human interpretation and exegesis. How would the algorithm apprehend such aspects? The knowledge gap to be filled is conceived from the question posed in the research, and that is a characteristic of the human.

Chart 5
Categories and subcategories in correlation with the admissibility index of Artificial Intelligence – Creation, Organization and Writing in articles, dissertations and PhD theses.

In the perspective to be shown next, there are type of "conditioned admissibility" not directly referring to the "types of texts" in terms of content. The "writing" category is present in two registers, in one of the cases it is associated to "assessment" by the subject — tool user — of the algorithm functionalities; and in the other case, in an isolated manner, to the use of the tool by people with disabilities. The graduate student admitted, in a direct answer, to use AI in her own writing. A student from the Professional Master's, however, expressed reservations to its use. She then justified her use of AI in her own writing in order to get to know the tool and evaluate its functionality so she could decide if she would use it or not. Furthermore, under the "writing" category, she stated the following: "The only positive point would be for people with social, emotional and cognitive maturity who would know how to use the tool to their benefit, but, once more: the principal of thinking, writing, criticism [added emphasis]" (Talitha, 47, Psychology, Mp). The content of the student's interview shows subjective aspects, intrinsically connected to singularity and socio-emotional development towards decision making. It attributes the decision power to use or not use AI-COW to the subject. The position of the Psychology master's student (who works as a therapist)12 12 Information about the mastering student's professional performance was found in her resume. presents correlated variables to the one belonging to the PhD student. In future studies, it would be interesting to investigate the schooling, professional field of work, and position towards AI use in writing production variables.

Still in the "writing" category, a graduate student in the Academic Master's expressed "admissibility with reservations" to the use of AI by a "person with disability". In this case, AI would help, depending on the specific disability. In the same "writing" category, the student declared "non-admissibility" of AI since it would implicate a loss of the subject's protagonism as someone who belongs to a certain social and cultural upbringing, and "the idea of writing and language as constitutive of my thought would be lost… we would end up losing in social relations … at the same time it would transform, it would remove all our constitutive character (ethnic), our citizenship [added emphasis]" (Alya, 50, Education, Ma). At first, the student's position seems paradoxical, but it carries the prospect of promoting technology accessibility to people with disabilities. In our study's final classification, regarding the positions of graduate students towards AI-COW, this particular Master's student was included in "admissibility with reservations".

Some highlights in other emerging categories and subcategories: a. "ethics and collective morality": "ethics and collective morality" was the only category grouping which obtained the highest numeric index of subcategories of "conditioned admissibility" when compared to the "non-admissibility" subcategories, and, in the content of the interviews, these subcategories appear as principles, norms and criteria which should guide the use and assessment of AI-COW both in the writing process and the writing product, punishing also the deviations; b. the category with absolute predominance as "non-admissibility" was "plagiarism", which is the reason why we cite here a representative quote about the relation of AI and plagiarism: "We think we will be concerned with plagiarism, self-plagiarism and, then, we begin facing a system which is masking [added emphasis]" (Bellatrix, 32, Education, Mp); and c. the "non-admissibility of AI-COW", in what concerns any of its use, method, technique and research approach shows no numeric indicator for "admissibility with reservations". It is worth emphasizing that the content of the non-admissibility positioning is represented by the sense of "human" in Education research, in stages that are at times formed by field observation, listening, interviews, perceptions and experience interpretation.

LIMITATIONS AND CONCLUSIONS

Some limitations to this research have been observed and serve as alerts for future studies: the first was a lack of balance between the number of interviewed PhD students (4) in comparison to the number of students from the Academic Master's (7) and from the Professional Master's (6), while expanding the number of students for the interviews should also be considered; the second was methodological, since, with regard to the 84.5% of graduate students who, according to the grouping of frequency measures of the questionnaire answers, answering anonymously, said they would use AI in text production, it was not possible to identify them previously in order to question them more deeply in the interview about their knowledge of the AI writing tool and in which practical situations of academic writing they made use or would make use of it; the third limitation, the need for more quantitative data related to AI in the questionnaires, in order to associate and compare them with the qualitative data from the interviews. Even while noting these limitations, we consider as relevant having collected data from 133 students through the questionnaire and from 17 graduate students in the individual interviews, generating a set of data and information relevant to the subject's current importance and to other studies and research, during the time of the Covid-19 pandemic.

From the total of the sample, 88,6% of the graduate students do not know AI for writing. Nevertheless, even though they are unfamiliar with these technological resources, the questionnaire response indicated the aggregated percentage of a tendency to use by 84,5%. The data per interview, in a response positioned as "admissibility of use with reservations", reached 70,6%. By course level, the highest frequency measure for AI use in questionnaire data was "frequently", with 31.8% in the Professional Master's group. In an intermediate measure in the scale, occasional use reached 48.8% in the Academic Master's and 44.7% in the PhD Program. It is not possible to conclude there is a significant difference of AI-COW in the different levels of graduate programs, only a general predisposition of use by the students according to conditioned admissibility.

The positioning of graduate students as to "admissibility of use with reservations" varies according to specific categories, in what concerns the individual and the context. At times, the conditions to use AI in writing are located on an individual scale (person with disability, psychological dimension to unblock writing, evaluation of the algorithm's functionality and the use of AI in a reference source by the author, the algorithm user). At other times, they regard the academic context (ethics and collective morality, also reflected in the rules or criteria about the use of AI in the institution), among other aspects, which is distributed in different categories. Thus, capturing the meaning of AI-COW use in the production of written texts in Education cannot be done in the pure state of an isolated category, but in the multiple connections and meanings of them all.

Positioning with regard to "admissibility with reservations" for the use of the algorithm in Education research indicates a tendency for the risk of naturalization and reproduction of AI writing practices to imply a removal of the author from the context of academic production, affecting his/her originality, autonomous and creative thought, in case the institution does not establish norms and criteria for AI use. Furthermore, AI use in the writing of theses, dissertations and articles complexifies the typology of plagiarism. This complexity constitutes another layer to identify academic fraud due to its sophisticated configuration. It hinders or even prevents the identification by pieces of software which verify similarities among research sources.

We also find it important to emphasize the repercussion of the "hybrid text: AI and human" in the processes of appropriation of autonomous, critical and creative reading, in connection with the impacts of "artisanal writing". The concept of "artisanal writing" brings in itself the presence of the writer. That is, it has codes which guide the perceptions and practices of the author's writing in certain context and to certain ends. The "artisanal text" is characterized as the original fruition of the author, also with the intertextualities, with the identity mark of the author's own style. Concerning these aspects of the AI-COW use, data and information from our research point to the strength of ethics and morality as guiding aspects of academic integrity, working as regulators to be established collectively — in norms and criteria —, among which plagiarism punishment in case of academic fraud.

It is our belief that the same categories of ethics and morality will be mobilized by subjects of the academic community to decide between conditioned admissibility or non-admissibility, in a conflicted relation, in order to manage institutional definitions. The categories of morality and ethics will be used in the positioning of non-admissibility of AI-COW use, according to data from the research, exactly due to the projection of its absences as the sense of academic integrity in the process of knowledge production, accountability falling on the researcher, the human being.

In what regards non-admissibility, predominant in the "literature review" category, there are still the emerging subcategories of "exegesis and interpretation", "source reliability", "appropriation process through studying and writing" and the repercussions in an academic literature that will be impoverished by mechanical reproduction.

Also compelling of our attention, due to the spontaneous opinion expressed in some interviews, was the relation established between the perception of the AI phenomenon in writing in correlation with schooling and work experience in teaching, resulting in emerging beliefs and evaluations about the admissibility or non-admissibility of the tool, as shown in more evident responses such as the following: "They are going to be horrible texts, and it has to do with my schooling, the texts will be totally square, they will look all the same … and in my experience [added emphasis] which is mainly with translators [apps which translate texts]" (Adhara, 40, Languages, PhD) and "I think that, because I'm an artist, I am much more afraid [added emphasis]" (Pollux, 43, Arts, PhD). These punctual data suggest that the confrontation between this perception of the phenomenon and students' professional work also influences their belief in the admissibility or non-admissibility of AI use in academic writing. This is a correlation to be studied and looked at more deeply in future research studies, including the issue of the assessment of AI use in academic writing in majors different than Education, such as Computer Sciences, Health Sciences and Business and Administration.

Feelings of fear, danger and strangeness towards AI-COW use have been mentioned by the Education graduate students. We reiterate these feeling in quoting both Adhara and Pollux, respectively: "I am afraid [added emphasis]" and "it will be a tragedy… I am very afraid [added emphasis]". We will also include Lua's statement, when mentioning the content of a final course assignment written by AI: "You are in my committee. You read the perfect text and you know it was not me [who wrote it], I find it strange [added emphasis]" (Lua, 26, Education, Ma). There is also room for investigating the relation between AI use in writing in different schooling levels and the perception of this use by different subjects.

Should a new category of articles, theses and dissertations be created such as the one generated partially by AI, to be legitimately recognized, adding to the nature of knowledge conceived, produced, published and appropriated by society? Such a question was inspired in part by our interview with Sol, who, in fact, is opposed to the use of AI-COW: "I think that, maybe, we could create another category, since technology creates new situations that we could not have imagined years ago [added emphasis]" (Sol, 34, Education, Ma).

In order to approach this reality in which AI tends to be present in the process and production of research projects, dissertations, theses, articles and other types of academic products, it is of utmost importance that educational institutions, scientific periodicals, publishers, promoters academic events and research funding institutions understand the phenomenon and its implications. In this manner, it will be possible to implement information and guidance measures supporting knowledge and research on the topic and determining situations which should be sanctioned for inadequate use of AI in both academic writing process and product.

Finally, from this study, we can conclude that Graduation Programs in Education — the focus of our article — should establish institutional rules regarding the use of algorithms in articles, theses and dissertations, answering, among other questions resulting from our study: why and to what end should we need a writing algorithm to develop part of the knowledge production in Education? Nevertheless, the fact is that the conception, the nature, and the responsibility for knowledge resulting from research belong not to an algorithm, but to concrete human subjects.

  • 1
    Fictitious name, followed by age, major and school level (Ma — Academic Master's; Mp — Professional Master's and Doctorate — PhD).
  • 2
    The reference made in the title to a PhD dissertation is meant to represent a Graduate study written production, but in the paper's approach, we also include Master's theses and articles.
  • 3
    When adequate, AI-COW will correspond to the acronym used in the paper for Artificial Intelligence – Creation, Organization and Writing.
  • 4
    The Academic Master's Degree has the purpose of preparing the professionals for teaching and research, while the Professional Master's degree is targeted to capacity building, considering the study of techniques, processes, and topics responding to the demands of the job market. https://www.gov.br/capes/pt-br/acesso-a-informacao/perguntas-frequentes/sobre-a-cap. Access on: July 27, 2022.
  • 5
    Covid-19 is a disease caused by the coronavirus, which generated a pandemic of global scale. In Brazil, the first case was registered in February 2020, and the declaration of community transmission in the country took place in March of the same year (cf. Agência Brasil, 2021AGÊNCIA BRASIL. Primeiro caso de covid-19 no Brasil completa um ano. Brasília, 26 fev. 2021. Disponível em: https://agenciabrasil.ebc.com.br/saude/noticia/2021-02/primeiro-caso-de-covid-19-no-brasil-completa-um-ano. Acesso: 27 jul. 2022.
    https://agenciabrasil.ebc.com.br/saude/n...
    ).
  • 6
    Given the extension of the data from the questionnaire and the interviews, which included other research topics, in this paper we restricted ourselves to only two of the main questions directly related to the use of AI in text production.
  • 7
    The values in the same line which do not share the same subindex (a, b) are significantly different with p<0.05. The percentages in bold indicate cells with residues superior to 1.96 (positive association), and the percentages in italics indicate cells with residues inferior to -1.96 (negative association).
  • 8
    Admissibility with reservations in this paper bears the same meaning as conditioned admissibility.
  • 9
    The cooccurrences take place due to the association between two or more words or topics (Bardin, 2011BARDIN, Laurence. Análise de conteúdo. São Paulo: Edições 70, 2011.) which were present in the content of the interviews and appeared concomitantly or in other parts of the material analyzed, at times contradicting one another. That is, the graduate student "defends" the "artisanal" text while criticizing the AI generated "standard" or "hybrid" texts: "AI" and "human" showing up in the same sentence or in an answer to a subsequent question. Such data on a table would not be intelligible in relation to the numeric datum by level of course (Master's and PhD) or text typology.
  • 10
    The two graduate students who answered thar they would use AI-COW in their own text are included in this figure. The third positioning was presented when answering a hypothetical question on admmitance or nonadmittance of AI-COW use.
  • 11
    Information about professional performance was extracted from the interviewee's profile.
  • 12
    Information about the mastering student's professional performance was found in her resume.
  • Funding: From Prof. Carlos Lopes: To the Research Support Fund (Fundo de Amparo à Pesquisa - FAP) of the Federal District (Brazil), for its support in the realization of the research (Public Notice DPG/UnB n. 0008/2021). From Prof. Ruben Comas Forgas: This article is part of the IAPOST project, concession "RTI2018-098314-B-I00, funded by MCIN/AEI/ 10.13039/501100011033, by "ERDF A way to build Europe" as well as the Ibero-american Network of Research in Academic Integrity (Red Ibero-americana de Investigación en Integridad Académica) (www.red-ia.org), funded by AUIP.

ACKNOWLEDGEMENTS

To the coordinators of the professional and academic Post-Graduate Programs in Education (Programas de Pós-Graduação em Educação —PPGE) who, in 2021, supported the data collection by sending the questionnaire to their graduate students.

To the PhD student Sirlene Rodrigues and the Master's student Bruna Passos, from the research line of Education, Technologies and Communication (Educação, Tecnologias e Comunicação — ETEC) of the PPGE at the University of Brasilia (Universidade de Brasília — UnB), for the help in data collection, including their encouragement of participation by other students and divulging of this research in the context of Covid-19.

To the students taking part in the research.

REFERÊNCIAS

  • AGÊNCIA BRASIL. Primeiro caso de covid-19 no Brasil completa um ano. Brasília, 26 fev. 2021. Disponível em: https://agenciabrasil.ebc.com.br/saude/noticia/2021-02/primeiro-caso-de-covid-19-no-brasil-completa-um-ano Acesso: 27 jul. 2022.
    » https://agenciabrasil.ebc.com.br/saude/noticia/2021-02/primeiro-caso-de-covid-19-no-brasil-completa-um-ano
  • ARAÚJO, Marcelo. O uso de inteligência artificial para a geração automatizada de textos acadêmicos: Plágio ou meta-autoria? Logeion: Filosofia da Informação, v. 3, n. 1, p. 89-107, 2016. https://doi.org/10.21728/logeion.2016v3n1.p89-107
    » https://doi.org/10.21728/logeion.2016v3n1.p89-107
  • BARAKINA, Elena Y.; POPOVA, Anna V.; GOROKHOVA, Svetlana; VOSKOVSKAYA, Angela S. Digital technologies and artificial intelligence Technologies in education. European Journal of Contemporary Education, v. 10, n. 2, p. 285-296, 2021. https://doi.org/10.13187/ejced.2021.2.285
    » https://doi.org/10.13187/ejced.2021.2.285
  • BARBERO, Álvaro. Escritura a la carta -Inteligencia artificial y literatura: la máquina que escribe. [Entrevista cedida ao] Canal 24h da TVE "Zoom Net". Reprodução da entrevista no sítio eletrônico do Instituto de Enginiería del Conocimiento, [s.l.], 8 fev. 2018. Disponível em: https://www.iic.uam.es/noticias/inteligencia-artificial-literatura-la-maquina-escribe Acesso em: 27 jul. 2022.
    » https://www.iic.uam.es/noticias/inteligencia-artificial-literatura-la-maquina-escribe
  • BARDIN, Laurence. Análise de conteúdo São Paulo: Edições 70, 2011.
  • COMAS FORGAS, Rubén.; CERDÀ NAVARRO, Antoni.; OLIVER TROBAT, Miquel.; SUREDA NEGRE, Jaume. Contract cheatin in Spain: identificación and analysis of academic "ghost-writing". In: GÓMEZ GARCÍA, Geraldo.; NAVAS-PAREJO, Magdalena Ramos.; RODRÍGUEZ JIMÉNEZ, Carmen.; DE LA CRUZ CAMPOS, Juan Carlos (Eds). Teoría y práctica en investigación educativa: una perspectiva internacional. Madrid: Dykyson, S.L, 2021. p. 473-485. Disponível em: https://iapost.uib.es/a/f31e9a9c514ae99e127c1e0982b5e860f20ef481/9788413771748_libro_dykinson_capitulo.pdf Acesso em: 27 jul. 2022.
    » https://iapost.uib.es/a/f31e9a9c514ae99e127c1e0982b5e860f20ef481/9788413771748_libro_dykinson_capitulo.pdf
  • COORDENAÇÃO DE APERFEIÇOAMENTO DE PESSOAL DE NÍVEL SUPERIOR. Sobre a CAPES 05 jan. 2021. Disponível em: https://www.gov.br/capes/pt-br/acesso-a-informacao/perguntas-frequentes/sobre-a-cap Acesso em: 27 jul. 2022.
    » https://www.gov.br/capes/pt-br/acesso-a-informacao/perguntas-frequentes/sobre-a-cap
  • FENG, Shihui; LAW, Nancy. Mapping artificial in education research: a network based keyword analysis. International Journal of Artificial Intelligence in Education, v.31, n. 2, p. 277-303, 2021. https://dx.doi.org/10.1007/s40593-021-00244-4
    » https://dx.doi.org/10.1007/s40593-021-00244-4
  • FYFE, Paul. How to cheat on you final paper: Assigning AI for student writing. AI & Society: Knowledge, Culture and Communication, London, 10 mar. 2022. https://doi.org/10.1007/s00146-022-01397-z
    » https://doi.org/10.1007/s00146-022-01397-z
  • LOPES, Carlos. O plágio como questão pública. In: NOVO, Ana; NOBEL, Ana; SIMÃO, João; PEREIRA, Pedro (orgs.). Plágio e integridade acadêmica na sociedade da informação v. 10. Lisboa: UAB Portugal, 2020. p. 15-33. https://doi.org/10.34627/ftpm-hq41
    » https://doi.org/10.34627/ftpm-hq41
  • OTSUKI, Grant Jun. Ok computer: To prevent students cheating with AI text-generators, we should bring them into the classroom. [S.l.]: The Conversation, 23 jan. 2020. Disponível em: https://theconversation.com/ok-computer-to-prevent-students-cheating-with-ai-text-generators-we-should-bring-them-into-the-classroom-129905 Acesso em: 27 jul. 2022.
    » https://theconversation.com/ok-computer-to-prevent-students-cheating-with-ai-text-generators-we-should-bring-them-into-the-classroom-129905
  • PARKER'S, P. Philip Parker's trick for authoring over 1 million books: don't write. [Entrevista cedida a] Bianca Bosker. The Huffington Post, [s.l.], 11 fev. 2013. Disponível em: https://www.huffpost.com/entry/philip-parker-books_n_2648820 Acesso em: 27 jul. 2022.
    » https://www.huffpost.com/entry/philip-parker-books_n_2648820
  • PINHO, Cíntia Maria de Araújo; MOURA, Amanda Ferreira de; GASPAR, Marcos Antonio; NAPOLITANO, Domingos Márcio Rodrigues. Identificação de deficiências em textos educacionais com a aplicação de processamento de linguagem natural e aprendizado de máquina. ETD – Educação Temática Digital, [s.l.], v. 24, n. 2, p. 350-372, abr./jun. 2022. https://doi.org/10.20396/etd.v24i2.8660061
    » https://doi.org/10.20396/etd.v24i2.8660061
  • RADFORD, Alec; WU, Jeffrey; AMODEI, Dario; AMODEI, Daniela; CLARK, Jack; BRUNDAGE, Miles; SUTSKEVER, Ilya. Better language models and their implications [Original post]. [S.l.]: OpenAI, 14 fev. 2019. Disponível em: https://openai.com/blog/better-language-models/ Acesso em: 27 jul. 2022.
    » https://openai.com/blog/better-language-models/
  • REWRITE GURU. Reescrever texto. Disponível em: https://rewriteguru.com/pt/ Acesso em: 27 jul. 2022.
    » https://rewriteguru.com/pt/
  • SAYURI, Juliana. Aplicativos que reescrevem artigos acendem alerta na acadêmica. Folha de São Paulo, São Paulo, ano 1-2, n. 34.130, 7 abr. 2019. Ilustríssima, 3.
  • SELWYN, Neil; HILLMAN, Thomas; EYNON, Rebecca; FERREIRA, Giselle; KNOX, Jeremy; MACGILCHRIST, Felicitas; SANCHO-GIL, Juana M. What's next for Ed-Tech? Critical hopes and concerns for the 2020s. Learning, Media and Technology, [S.l.], v. 45, n. 1, p. 1-6, 2020. https://doi.org/10.1080/17439884.2020.1694945
    » https://doi.org/10.1080/17439884.2020.1694945
  • SHARPLES, Mike. New AI tools that can write student essay require educators to rethink teaching and assessment. [S.l], 17 maio 2022. Disponível em: .https://blogs.lse.ac.uk/impactofsocialsciences/2022/05/17/new-ai-tools-that-can-write-student-essays-require-educators-to-rethink-teaching-and-assessment/ Acesso em: 27 jul.2022.
    » https://blogs.lse.ac.uk/impactofsocialsciences/2022/05/17/new-ai-tools-that-can-write-student-essays-require-educators-to-rethink-teaching-and-assessment/
  • SUREDA-NEGRE, Jaume; COMAS-FORGAS, Ruben; MUT-AMENGUAL, Bartomeu. Las "fábricas" de trabajos académicos: una incitación al fraude en las aulas [S.l]: Universitat de les Illes Balears, Grupo de Investigación Educación y Ciudadania, 2007. (Informes de investigación). Disponível em: https://www.researchgate.net/publication/216508231 Acesso em: 27 jul. 2022.
    » https://www.researchgate.net/publication/216508231
  • VICARI, Rosa Maria. Tendências em inteligência artificial na educação no período de 2017-2030: sumário executivo. Brasília: SENAI, SESI, 2018.

Publication Dates

  • Publication in this collection
    29 July 2024
  • Date of issue
    2024

History

  • Received
    12 Oct 2022
  • Reviewed
    25 Apr 2023
  • Accepted
    04 May 2023
ANPEd - Associação Nacional de Pós-Graduação e Pesquisa em Educação Rua Visconde de Santa Isabel, 20 - Conjunto 206-208 Vila Isabel - 20560-120, Rio de Janeiro RJ - Brasil, Tel.: (21) 2576 1447, (21) 2265 5521, Fax: (21) 3879 5511 - Rio de Janeiro - RJ - Brazil
E-mail: rbe@anped.org.br