Learning analytics and personalization of learning: a review

Análisis de aprendizaje y personalización del aprendizaje: una revisión

Análise de aprendizagem e personalização de aprendizagem: uma revisão

Abstract

Education in the 21 ^st century is increasingly mediated by digital technologies in a context in which enormous amounts of information are daily generated. Regarding this and considering the imminent application of emerging trends such as “Internet of Things” (IoT), the study of its educational effects becomes a matter of great relevance for both educational researchers and practitioners. In this context, “Learning Analytics” takes on special importance as a perspective to approach the aforementioned issue, especially from a very relevant topic: the personalization of learning. In this sense, a systematic review of literature about learning analytics published in the last two decades was carried out to identify its potential as a factor in strengthening the personalization of learning. The results show a set of key factors that include aspects related to assessment, the use of dashboards, social learning networks, and intelligent tutoring, and the importance of monitoring, feedback, and support.

21st-Century Skills; Pedagogical Issues; Information Literacy; Data Science Applications in Education; Evaluation Methodologies

Resumen

La Educación en el siglo XXI está cada vez más mediada por las tecnologías digitales en un contexto en el que se generan diariamente enormes cantidades de información. Con respecto a esto y considerando la aplicación inminente de tendencias emergentes como el “Internet de las cosas” (IoT), el estudio de sus efectos educativos se convierte en un asunto de gran relevancia tanto para los investigadores educativos como para los profesionales. En este contexto, “Learning Analytics” adquiere especial importancia como perspectiva para abordar el tema mencionado, especialmente desde un tema muy relevante: la personalización del aprendizaje. En este sentido, se llevó a cabo una revisión sistemática de la literatura sobre análisis de aprendizaje publicada en las últimas dos décadas para identificar su potencial como factor de fortalecimiento de la personalización del aprendizaje. Los resultados muestran un conjunto de factores clave que incluyen aspectos relacionados con la evaluación, el uso de paneles, redes de aprendizaje social y tutoría inteligente, y la importancia del monitoreo, retroalimentación y apoyo

Competencias del Siglo XXI; Problemas Pedagógicas; Alfabetización Informativa; Aplicaciones de la Ciencia de Datos en la Educación; Metodologías de Evaluación

Resumo

A Educação no século XXI está cada vez mais mediada pelas tecnologias digitais em um contexto em que enormes quantidades de informação são geradas diariamente. Nesse sentido, considerando a iminente aplicação de tendências emergentes, como a “Internet das Coisas” (IoT), o estudo de seus efeitos educacionais torna-se uma questão de grande relevância tanto para pesquisadores quanto para profissionais da Educação. Nesse contexto, o “Learning Analytics” adquire uma importância especial como uma perspectiva para abordar o tema supracitado, especialmente a partir de um tema muito relevante: a personalização da aprendizagem. Por isso, uma revisão sistemática da literatura sobre learning analytics publicada nas últimas décadas se torna importante para identificar o seu potencial como fator para fortalecer a personalização da aprendizagem. Os resultados mostram um conjunto de fatores chave que incluem aspectos relacionados à avaliação, utilização de quadros, redes sociais de aprendizagem e tutoria inteligente, e a importância do acompanhamento, feedback e apoio.

Competências do Século XXI; Problemas Pedagógicos; Literacia Informacional; Aplicações da Ciência de Dados na Educação; Metodologias de Avaliação

1 Introduction

The digital transformation generated by the intensive appropriation of Information and Communication Technologies (ICT) specifically of the Internet, has brought about changes in the processes of interaction between people, which has caused important transformations in all sectors of human life, including, of course, the Education (Saif et al. , 2022).

In this context, some theoretical instances of mass production and consumption of information have begun to materialize in products and services that are gradually becoming more widely available. One of these instances, which is known as the “Internet of Things” (IoT) has begun to articulate the mass production and interconnection of everyday objects with each other through information networks, so that is considered one of the main sources of information for a global phenomenon called “Big Data” (Ferrão; Prata; Alves, 2020).

Considering the above, it should be mentioned that in most cases the IoT and the most common applications of Big Data would have the main function of putting the management of huge amounts of data at the service of improving user experiences in multiple fields of human activity (Konstantinidis, 2021). About this, Martinez-Maldonado et al . (2018) indicate that the IoT allowed to advance in the understanding of the complex relationships between people, objects, space, and time in both physical and digital environments and to understand that this interconnection includes “data” as a critical actor to make visible the invisible in such relationships.

In the field of Education, the improvement of user experiences through the application and use of Big Data would focus on what is known as “Learning Analytics” (LA), which is considered an emerging field of educational research.

LA is a new interdisciplinary field that takes advantage of learning activities captured as data and stored within digital learning environments such as MOOCs. These data can be “mined” and analyzed through digital traces (log data) to identify patterns of learning behaviors and provide insights into learning practices (Xu; Wu; Ouyang, 2023). The above includes identifying potential dropouts of a course based on predictive modeling, using visualization techniques, and providing instructors and mentors with overviews of learners’ activities.

Also, LA includes techniques such as predictive models, profile construction, adaptive learning, optimization of learning success, interventions, analysis of social networks, and feelings (Waheed et al ., 2023).

In this regard, Chatti and Muslim (2019, p. 244) indicate that:

Learning analytics (LA) can play an important role by analyzing data collected from various learning environments, supporting customized activities that meet the different learners’ needs and goals, as well as providing insights and understanding into how learners perform in these environments and how to best support this process.

As shown in Figure 1, it is from 2010 that an exorbitant growth process begins in the publication of research related to LA. This denotes a growing interest on the part of the academic community not only in issues related to the educational use of ICT but specifically in those that involve very advanced management of information, among which is situated the “Artificial intelligence” (Sijing; Lan, 2018), the “Machine learning” (Chai; Lin; Li, 2018), the “educational data mining” – EDM (Doko; Bexheti, 2018) and of course the LA”.

Figure 1
Research on learning analytics based on Scopus data

Among the great diversity of positions and focuses of research on these issues, many researchers agree that such topics can provide significant benefits to Education as they help to develop student-centered processes and provide data and tools that both educational institutions and teachers could use to make real-time predictions about the behavior and academic performance of students or to implement contingency plans for the improvement of teaching and learning in digital environments (Aldowah; Al-Samarraie; Fauzy, 2019).

On the other hand, one of the promises of the use of technologies in Education since its inception has had to do with the promotion of personalization schemes for learning (Yi et al ., 2017). This issue is recognized as very important for 21st-century Education (Ament; Edwards, 2018), however, the educational reality in terms of personalization of learning, especially in developing countries, is very distant from the fulfillment of that promise.

Considering the above, LA has begun to be conceived as a channel for the generation of personalized learning experiences (Muslim et al ., 2017), however, there is still very few research generated on this matter, according to what is shown in Table 1.

Table 1
Scopus results on: “Learning Analytics” AND “Personalized Learning”

Year	# Published Papers	Year	# Published Papers
2011	1	2017	18
2012	1	2018	28
2013	6	2019	32
2014 2015	6 8	2020 2021	33 25
2016	19	2022	31

Source: Own elaboration based on Scopus data (2023)

Learning personalization involves building student profiles and adapting learning environments, consequently improving relevance and productivity in Education; for this reason, it is considered part of the vanguard of educational reform (Verbert et al ., 2013).

Because the personalization of learning generated by the use of new technologies must demand changes in teaching, learning, and evaluation, the promise of LA to revolutionize educational institutions and processes, is very interesting to address, like never before. For this purpose, a systematic literature review has been carried out on studies published in the last 10 years on this subject to understand how LA can become a determining factor for the construction of personalized learning experiences.

2 Method

According to Hart (2018), literature reviews play a key role within the methodological spectrum in research, since they allow, if carried out with adequate levels of quality, that is, in terms of its wideness, depth, rigor, consistency, clarity and brevity, present an adequate synthesis of the research development on a certain topic.

The review was conducted following the recommendations of Higgins and Green (2011), who propose five major stages, whose detail is expressed in Figure 2.

Figure 2
Review method

2.1 Review question and inclusion-exclusion criteria

The review process began with the approach of a guiding question: what aspects of LA contribute to the personalization of learning? “It was established as a criterion that only articles that presented the results of studies on LA would be reviewed. For this purpose, a string of searching descriptors was defined that would produce results broad enough to obtain answers from the review. The descriptors used were “TITLE-ABS-KEY (“learning analytics”)”, which were applied to search in title, abstract, and keywords in Scopus, which would guarantee an adequate level of quality in terms of the sources of consult and their coverage.

2.2 Searching and selection of studies

In this stage, the first search was made in Scopus, which yielded 5,651 documents. The first filtering by type of document (articles and conference papers) and by area of knowledge to “Social Sciences”, reduced the set of documents to 2,425 items and modified the string of search descriptors like this: TITLE-ABS-KEY(“learning analytics”) AND (LIMIT-TO (DOCTYPE,“cp”) OR LIMIT-TO (DOCTYPE,“ar”)) AND (LIMIT-TO (SUBJAREA,“SOCI”)).

To generate a manageable set of texts, a representative sample (n = 332) with a confidence level of 95% and an error of 5% was calculated.

With this final set of texts, an abstracting process was carried out with the results ordered by year and highest citations, through which the articles that effectively presented research results and thematic relevance could be identified. As a result, 100 full-text articles were obtained, in a weighted manner and proportional to the number of publications per year, which were recorded in a documentation matrix.

2.3 Data analysis

In this stage, the in-depth reading of the selected texts was carried out, a process in which some key ideas were extracted and recorded. The analysis was carried out in a documentation matrix where the following data were stored: the text reference, the year of publication, and the key factors related to the personalization of learning. From the reviewed articles, fragments of text were extracted, which were processed through QDA software, whose results were presented through processes of frequency counting and percentage of associated codes. As a triangulation process, 12 key factors were standardized or unified to perform an analysis of the frequency of appearance.

2.4 Analysis of report bias

The management of the bias in the data analysis process was conducted based on the observation and interpretation of the same data set by two observers, from whose results a coefficient of Cohen’s Kappa was applied (k = 0,512), whose result, according to Tang et al. (2015), allows us to recognize an adequate level of inter-rater valuation.

2.5 Presentation and interpretation of the results and obtaining the conclusions

The last stage of the review was to synthesize the results and write the review report based on an IMRaD structure.

3 Results

As a result of the data analysis and extraction process, the 12 key LA factors related to the personalization of learning are presented in Table 2.

Table 2
Key factors for learning personalization

Key factors	Count	% Codes
Continuous and formative assessment	55	22.36%
Monitoring, feedback, and support	41	16.67%
Dashboards	31	12.60%
Types of data	26	10.57%
Intelligent tutoring	16	6.50%
Self-regulated learning	15	6.10%
Game-based learning assessment	10	4.07%
Social learning networks	10	4.07%
Warning system	7	2.85%
Learning styles	4	1.63%

Source: Own elaboration (2023)

3.1 Continuous and formative assessment

According to Albelbisi, Yusop and Salleh (2018), the assessment carried out periodically and constantly is an important factor in determining the future success of a group of students or each of its members.

Researchers have explored various evaluation techniques for assessing the quality of digital learning environments. These include hetero-evaluation, self-evaluation, and peer evaluation, with the completion of weekly tests showing the strongest correlation with final exam results (Admiraal; Huisman; Pilli, 2015). Burrows and Kumar (2018) suggest that assessing learning progress through LA may be more beneficial than a final grade, as it allows for more detailed and personalized measurement and correlation with other student activities.

In summary, within the framework of LA, continuous assessment variables were repeatedly chosen as the most important variables by all prediction models, including those related to self-assessment.

Other studies that mention continuous and formative assessment as a personalization factor of learning are: Fernandez-Nieto, Echeverria and Shum 92021), Holmes et al . (2018), Howard, Meehan and Parnell (2018), Kurilovas (2019) and Nguyen et al . (2017).

3.2 Monitoring, feedback, and support

This key factor in the personalization of learning is essentially related to the role of the teacher, but some studies indicate that it can also be carried out automatically by a system. As an example of the above considerations, Xing and Du (2019) show a personalization mechanism implemented through an automated agent in intelligent systems that supports the design and delivery of personalized weekly interventions as a way to calculate the probability of dropping out of individual students. In this regard, it is indicated that the instructors who provide personalized assistance still have limitations given the time and effort required (Rienties; Cross; Zdrahal, 2017), which can be corrected with an intelligent agent that generates predefined interventions by expert teachers or based on historical data. The above can be specified through an automated email, an SMS, or even a video message delivered to the student.

Other studies that mention monitoring, feedback, and support as factors of personalization of learning are Conde et al. (2018), Goh et al . (2018), Liu et al . (2017), Tempelaar et al . (2018) and Worsley, Martinez-Maldonado and D’Angelo (2021).

3.3 Dashboards

Dashboards are conceived as user interfaces that allow the graphic visualization of data generated by the user, involving elements of social learning and generating continuous feedback from such data. Being one of the most mentioned key factors in the reviewed articles, these visualizations have originated along with the second generation of LA that focuses more on being descriptive than predictive.

As an example, Arnold and Pistilli (2012) use a panel of traffic lights in the Dashboards to identify the progress of the students, with which they can identify if it is satisfactory, with slight or urgent risk, and thus activate contingency plans on behalf of the teachers. An educational dashboard should systematically provide timely, continuous easily visualized data on student´s performance (Boscardin et al. , 2018).

In synthesis, the dashboards in LA help users collect personal information about various aspects of their lives, behavior, habits, thoughts, and interests. Likewise, they help to improve self-knowledge by providing tools for the review and analysis of their processes (Verbert et al ., 2013, p. 2).

Other studies that mention dashboards as personalization learning factors are: Boscardin et al . (2018), Clow (2013), Ellaway et al . (2014), Khosravi et al . (2021), Liu et al. (2017) and Williams (2017).

3.4 Types of data

Much of the success of LA lies in the data and its quality, for this reason, it was considered relevant to identify the types of data that are significant and essential for the personalization of learning. Each of the following authors tells us about different types of information that can be collected through LA tools or techniques.

Regarding this, Albelbisi, Yusop and Salleh (2018) refer to three dimensions of data; The first one, called “presage”, includes the demographic data of the student, motivation and interactivity, and data of the instructor. The second dimension is known as “process”, which includes pedagogy factors, engagement patterns, instructional design, evaluation, plagiarism, and sustainability. Finally, the third dimension is the “product”, which includes quality factors and school dropout rates.

On the other hand, Zhang et al . (2018) included the following variables: number of accesses, online working time, navigation resources, frequency of access, publications in forums, dispatch of tasks, access period, and test attempts.

3.5 Intelligent tutoring

Within the reviewed studies, the learning environments that have had an approach to personalization have technological mediation regardless of their teaching-learning modality. Most of the research has been carried out in massive environments called MOOCs (46,84%) whose name means “Massive Open Online Courses”, or in learning environments designed for the application of serious games (22.28%).

According to Zhang et al . (2018), the environments characterized by this personalized approach are defined as “intelligent learning environments” that can offer instantaneous and adaptable support to students through the immediate analysis of their needs from different perspectives.

Lajoie and Azevedo (2012, p. 808) refer to intelligent tutoring, which is an electronic system that seeks to improve learning that “must possess: (a) knowledge domain (expert model), (b) knowledge of the learner (student model), and (c) knowledge of teaching strategies (tutor)”.

Other studies that mention intelligent tutoring as a personalization learning factor are Hwang (2014), Kato, Kambayashi and Kodama (2018) and Shemshack, Kinshuk and Spector (2021).

3.6 Self-regulated learning

The personalization of learning can be achieved if the learning environments are developed from pedagogical designs based on self-regulated learning (Romero et al., 2019). Without an adequate pedagogical foundation, customization will be very difficult to achieve, even when sophisticated technological means are applied, including LA.

In this sense, if the learning activities have been designed for the student to generate information about their learning process, which can then be captured, analyzed, and interpreted, then the entire educational process can be powered by the LA techniques. According to Engeness and Mørch (2016), the data generated through learning activities are the main data sources for adaptive feedback systems.

In a complementary way, Singh and Mørch (2018) indicate that to further improve student learning, teachers and educational technologies should incorporate comments much more actively into learning activities.

In summary, Ifenthaler and Schumacher (2016) state that differences in learning success are attributed predominantly to the self-regulation abilities of students that are relevant to initiating and sustaining learning processes. Therefore, students expect that LA allows them to support their planning and organization of learning, perform self-assessments, offer adaptive recommendations, and produce personalized analyses of their learning activities.

Other studies that mention self-regulated learning as the pedagogical basis for learning personalization are Saqr, Peeters and Viberg (2021), Choles (2016), Slade and Prinsloo (2013), Thompson and Cook (2017), Vives-Varela et al . (2014).

3.7 Game-based learning assessment

In addition to what has already been mentioned concerning the relevance of continuous and formative assessment, the specific focus of game-based learning assessment is highlighted as a factor that generates data that can be analyzed and that reflects personal circumstances and conditions that affect learning.

In this regard, Terras et al . (2018) pointed out the relevance of serious educational games as the basis for the design of student-centered learning environments, which allow the data to be collected constantly to generate the features of personalization.

From this perspective, Rowe et al . (2017) confirm the advantages of registering all game events, which are grouped into five main categories: (a) Location/Vector player particle movement; (b) Time and location of the impulses; (c) Number and location of other particles; (d) General features of the game; and (e) Result of the game. By storing all this data, the analysis derived from LA is allowed to generate better relationships that produce more personalization.

Other studies that mention game-based learning assessment as a key factor for learning personalization are Gee (2003), Haladyna and Downing (2004), Hersh and Leporini (2012), Shute et al . (2010), and Thomas and Brown (2011).

3.8 Social learning networks

An issue that is striking as a result of the review of published studies about LA has to do with the relationship between social interaction and personalized learning. Although a first glance they seem two contradictory issues -social and personal-, within the framework of LA they have a close relationship, which is materialized in the scope of social interaction (in digital environments) and yields a lot of data related to the journeys an individual travels as part of their learning paths (Buitrago; Chiappe, 2019).

In this sense, Sunar et al . (2016) point out the existence of what is known as “Social Learning Networks”, understood as the set of connections between people, media, agents, institutions, and resources organized to achieve learning goals or objectives. From the perspective of LA, these networks must have navigation support, recommendation, and information search services to support personalized and lifelong learning.

Other studies that mention “Social Learning Networks” as a key factor for learning personalization are Doleck, Lemay and Brinton (2021), Williams, Kim and Keegan (2015), and Williams (2017).

3.9 Warning systems

The systems of educational recommendation and prevention (warning systems) are defined as “any system that produces individualized recommendations or that has the effect of guiding the user in a personalized way to interesting or useful objects in a large space of possible options” (Vekariya; Kulkarni, 2012, p. 1).

This key factor is connected to some of the previous ones because the prevention and warning systems began with the idea of identifying and helping the students who were at risk and improving retention/dropout rates in the learning processes. These systems use the results obtained from data analysis and Dashboards to send alerts to students with recommendations and appropriate interventions sometimes developed by the teacher or on other occasions automatically generated by a platform.

An example of the above is stated by Kurilovas (2019) who creates and implements customized learning units (UoL) suitable for specific students according to their personal needs. The recommendation system created recommends students the most appropriate learning components to compose the UoL according to the student profile generated from the identification of their learning style.

3.10 Learning styles

Although learning styles are one of the least mentioned key factors in the studies reviewed on LA, it carries a special interest in this regard, given the affinity of this topic in the context of personalization of ICT-mediated learning.

Tempelaar et al . (2018) define personalized learning environments taking into account the application of a framework of learning styles previously addressed in Fasihuddin, Skinner and Athauda (2015) and Felder and Silverman (1988). This framework provides adaptive navigation support through the classification and concealment of learning materials based on student’s learning styles and preferences (Singh; Mørch, 2018). From this perspective, personalized learning can be built from the generation of the student’s profile comprehensively including information such as knowledge, interests, objectives, cognitive traits, type of learning behavior, etc. (Tempelaar et al. , 2018).

Other studies that mention Learning Styles as a key factor for learning personalization are Jena (2018), Mothukuri et al. (2017), Pappas, Giannakos and Sampson (2019), and Soler Costa et al . (2021)

4 Discussion

According to the specialized literature, the personalization of learning is achieved through the construction of a detailed profile of the student and the personalization of both the study material and the learning environment according to their tastes, preferences, skill levels, and the field of knowledge to learn.

In the field of 21st-century Education, learning environments must be intelligent and provide the teacher with individual information about each student; an expert domain must be established to determine the level of competence and its development in each one of them, together with strategies to encourage them to reach the expert level. These environments require Dashboards, warning systems, systems of interventions by the teacher, and coherent and pertinent evaluation systems that allow collecting and generating useful information to self-assess and understand their weaknesses and strengths. Such systems must also allow access to reinforcement material and information that allows the teacher to give formative feedback, constant monitoring, and timely support.

The evidence extracted from the reviewed studies allows us to recognize that only a few learning experiences mediated by ICT have managed to apply complete personalization strategies, most of them in MOOCs.

Within this context of learning, the relevance of the application of serious games is recognized, as a mechanism for detecting personalized information that covers several aspects of the learning process and that can be analyzed and interpreted later for the benefit of the same student. In this regard, the inherent complexity of the process of capturing, storing, visualizing, and interpreting the data and the need to ensure its relevance and quality remain as a reflection since if this double condition were not met, the data extracted would not be enough to generate reliable and useful LA processes.

LA has become a social process that aims to develop learning personalization. From the digital activity of students, information is captured and analyzed, especially in social networks and virtual communities about their behavior, personality, interests, and motivations, information that cannot be obtained from other sources, which allows for generating more significant data that can be analyzed by predictive models that will ultimately be the basis for building personalization strategies.

With the incorporation of LA into the dynamics of Education in digital environments, the design of the learning process is restructured incorporating continuous improvements concerning assessment and feedback, but always under a certain sense of paradox, in which at the micro level, the learning environment can be customized for one particular student, but at the macro level there is a risk of falling back into standardization, since all students would be sheltered by the same technologies, codes, and algorithms.

5 Acknowledgments

We thank Universidad de La Sabana (Technologies for Academia - Proventus Research Group (EDUPHD-20-2022 Project)) for the support received for the preparation of this article.

References

• ADMIRAAL, W.; HUISMAN, B.; PILLI, O. Assessment in massive open online courses. Electronic Journal of e-Learning , [s. l.], v. 13, n. 4, p. 207-216, 2015.
• ALBELBISI, N.; YUSOP, F. D.; SALLEH, U. K. M. Mapping the Factors Influencing Success of Massive Open Online Courses (MOOC) in higher education. Eurasia Journal of Mathematics, Science and Technology Education , Eastbourne, v. 14, n. 7, p. 2995-3012, 22 May 2018. https://doi.org/10.29333/ejmste/91486
  » https://doi.org/10.29333/ejmste/91486
• ALDOWAH, H.; AL-SAMARRAIE, H.; FAUZY, W. M. Educational data mining and learning analytics for 21st century higher education: a review and synthesis. Telematics and Informatics, [s. l.], v. 37, p. 13-49, Apr. 2019. https://doi.org/10.1016/j.tele.2019.01.007
  » https://doi.org/10.1016/j.tele.2019.01.007
• AMENT, V.; EDWARDS, R. Better teaching and more learning in mobile learning courses: Towards a model of personable learning. In: 14TH INTERNATIONAL CONFERENCE ON MOBILE LEARNING, 14., 2018, Lisbon. Proceedings[...] Lisboa: IADIS, 2018. p. 214-218. Available from: https://www.scopus.com/inward/record.uri?eid=2-s2.0-85052241220&partnerID=40&md5=6ca1258e009575fc0f41ba59f5ac8718 . Access in: 2020 July 10.
  » https://www.scopus.com/inward/record.uri?eid=2-s2.0-85052241220&partnerID=40&md5=6ca1258e009575fc0f41ba59f5ac8718
• ARNOLD, K. E.; PISTILLI, M. D. Course signals at Purdue: using learning analytics to increase student success. In: INTERNATIONAL CONFERENCE ON LEARNING ANALYTICS AND KNOWLEDGE, 2., Vancouver, 2012. Proceedings[...] Vancouver: ACM Press, 2012. p. 267-270. Available from: http://dl.acm.org/citation.cfm?doid=2330601.2330666 Access in: 2019 Jun 18.
  » http://dl.acm.org/citation.cfm?doid=2330601.2330666
• BOSCARDIN, C. et al. Twelve tips to promote successful development of a learner performance dashboard within a medical education program. Medical Teacher, London, v. 40, n. 8, p. 855-861, ago. 2018. https://doi.org/10.1080/0142159X.2017.1396306
  » https://doi.org/10.1080/0142159X.2017.1396306
• BUITRAGO, M.; CHIAPPE, A. Representation of knowledge in digital educational environments: a systematic review of literature. A ustralasian Journal of Educational Technology, Tugun, v. 35, n. 4, p. 46-62, 2019. https://doi.org/10.14742/ajet.4041
  » https://doi.org/10.14742/ajet.4041
• BURROWS, J.; KUMAR, V. The objective ear: assessing the progress of a music task. In: CHANG, M. et al. (Eds.). Challenges and solutions in smart learning. Singapore: Springer, 2018. p. 107-112.
• CHAI, M.; LIN, Y.; LI, Y. Machine learning and modern education. In: INTERNATIONAL CONFERENCE, eLEOT 2018, Shangai, 2018. Proceedings[…]. e-learning, e-education, and online training, Shanghai: Springer International. p. 41-46.
• CHATTI, M. A.; MUSLIM, A. The PERLA framework: blending personalization and learning analytics. International Review of Research in Open and Distance Learning, Athabasca, v. 20, n. 1, p. 243-261, 2019. https://doi.org/10.19173/irrodl.v20i1.3936
  » https://doi.org/10.19173/irrodl.v20i1.3936
• CLOW, D. An overview of learning analytics. Teaching in Higher Education, [s. l.], v. 18, n. 6, p. 683-695, Aug. 2013. https://doi.org/10.1080/13562517.2013.827653
  » https://doi.org/10.1080/13562517.2013.827653
• CONDE, M. A. et al. Teamwork assessment in the educational web of data: a learning analytics approach towards ISO 10018. Telematics and Informatics, [s. l.], v. 35, n. 3, p. 551-563, June 2018. https://doi.org/10.1016/j.tele.2017.02.001
  » https://doi.org/10.1016/j.tele.2017.02.001
• DOKO, E.; BEXHETI, L. A. A systematic mapping study of educational technologies based on educational data mining and learning analytics. In: 2018 7th MEDITERRANEAN CONFERENCE ON EMBEDDED COMPUTING (MECO), 7 th, 2018, Budva, Montenegro. Proceedings[...]. Budva: IEEE, 2018. Available from: https://ieeexplore.ieee.org/document/8406052/ Access in: 2019 Apr 10.
  » https://ieeexplore.ieee.org/document/8406052/
• DOLECK, T.; LEMAY, D. J.; BRINTON, C. G. Evaluating the efficiency of social learning networks: Perspectives for harnessing learning analytics to improve discussions. Computers & Education, [s. l.], v. 164, p. 104-124, Apr. 2021. https://doi.org/10.1016/j.compedu.2021.104124
  » https://doi.org/10.1016/j.compedu.2021.104124
• ELLAWAY, R. H. <italic>et al</italic> . Developing the role of big data and analytics in health professional education. Medical Teacher , Edingurgh, v. 36, n. 3, p. 216-222, Mar. 2014. https://doi.org/10.3109/0142159X.2014.874553
  » https://doi.org/10.3109/0142159X.2014.874553
• ENGENESS, I.; MØRCH, A. Developing writing skills in english using content-specific computer-generated feedback with essaycritic. Nordic Journal of Digital Literacy, [s. l.], v. 10, n. 2, p. 118-135, June 2016. https://doi.org/10.18261/issn.1891-943x-2016-02-0
  » https://doi.org/10.18261/issn.1891-943x-2016-02-0
• FASIHUDDIN, H.; SKINNER, G.; ATHAUDA, R. A framework to personalise open learning environments by adapting to learning styles In: INTERNATIONAL CONFERENCE ON COMPUTER SUPPORTED EDUCATION, 7., Lisbon, 2015. Proceedings[…]. Lisbon: SCITEPRESS - Science and and Technology Publications, 2015. Available from: http://www.scitepress.org/DigitalLibrary/Link.aspx?doi=10.5220/0005443502960305 Access in: 2019 June 19.
  » http://www.scitepress.org/DigitalLibrary/Link.aspx?doi=10.5220/0005443502960305
• FELDER, R. M.; SILVERMAN, L. K. Learning and teaching styles in engineering education. Engineering Education, [s. l.], v. 78, n. 7, p. 674-681, 1988.
• FERNANDEZ-NIETO, G. M., ECHEVERRIA, V, SHUM, S. R. Storytelling with learner data: guiding student reflection on multimodal team data. IEEE Transactions on Learning Technologies, v. 14, n. 5, p. 695-708, Oct. 2021. https://doi.org/10.1109/TLT.2021.3131842 Era et al.
  » https://doi.org/10.1109/TLT.2021.3131842
• FERRÃO, M. E.; PRATA, P.; ALVES, M. T. G. Multiple imputation in big identifiable data for educational research: an example from the Brazilian education assessment system. Ensaio: Avaliação e Políticas Públicas em Educação , Rio de Janeiro, v. 28, n. 108, p. 599-621, July./Sep. 2020. https://doi.org/10.1590/S0104-40362020002802346
  » https://doi.org/10.1590/S0104-40362020002802346
• GEE, J. P. What video games have to teach us about learning and literacy Basingstrokee: Palgrave Macmillan and Houndmills, 2003.
• GOYAL, T. et al. Big data handling over cloud for internet of things. International Journal of Information Technology and Web Engineering, [s. l.], v. 13, n. 2, p. 37-47, Apr. 2018. https://doi.org/10.4018/IJITWE.2018040104
  » https://doi.org/10.4018/IJITWE.2018040104
• HALADYNA, T. M.; DOWNING, S. M. Construct-irrelevant variance in high-stakes testing. Educational Measurement: Issues and Practice, [s. l.], v. 23, n. 1, p. 17-27, 2004. https://doi.org/10.1111/j.1745-3992.2004.tb00149.x
  » https://doi.org/10.1111/j.1745-3992.2004.tb00149.x
• HART, C. Doing a literature review: releasing the research imagination. London: Sage, 2018.
• HERSH, M. A.; LEPORINI, B. Accessibility and usability of educational games for disabled students. In: GONZALEZ, C. (Ed.). Student usability in educational software and games: improving experiences: improving experiences. Hershey: IGI Global, 2012. p. 1-40.
• HIGGINS, J. P. T.; GREEN, S. Cochrane handbook for systematic reviews of interventions version 5.1.0. The Cochrane Collaboration, 2011. Available from: https://es.cochrane.org/sites/es.cochrane.org/files/public/uploads/manual_cochrane_510_web.pdf Access in: 2020 Aug 24.
  » https://es.cochrane.org/sites/es.cochrane.org/files/public/uploads/manual_cochrane_510_web.pdf
• HOLMES, M. et al. Near real-time comprehension classification with artificial neural networks: decoding e-learner non-verbal behavior. IEEE Transactions on Learning Technologies, [s. l.], v. 11, n. 1, p. 5-12, Jan. 2018.
• HOWARD, E.; MEEHAN, M.; PARNELL, A. Contrasting prediction methods for early warning systems at undergraduate level. The Internet and Higher Education, New York, v. 37, p. 66-75, Apr. 2018. https://doi.org/10.1016/j.iheduc.2018.02.001
  » https://doi.org/10.1016/j.iheduc.2018.02.001
• HWANG, G.-J. Definition, framework and research issues of smart learning environments: a context-aware ubiquitous learning perspective. Smart Learning Environments , v. 1, n. 1, 4, Dec. 2014. https://doi.org/10.1186/s40561-014-0004-5
  » https://doi.org/10.1186/s40561-014-0004-5
• IFENTHALER, D.; SCHUMACHER, C. Student perceptions of privacy principles for learning analytics. Educational Technology Research and Development , v. 64, n. 5, p. 923-938, Oct. 2016.
• JENA, R. K. Predicting students’ learning style using learning analytics: a case study of business management students from India. Behaviour & Information Technology, v. 37, n. 10-11, p. 978-992, 2 nov. 2018. https://doi.org/10.1007/s11423-016-9477-y
  » https://doi.org/10.1007/s11423-016-9477-y
• KATO, T.; KAMBAYASHI, Y.; KODAMA, Y. Using a programming exercise support system as a smart educational technology. In: USKOV, V. L. et al. (Eds.). Smart universities. Cham: Springer International, 2018. p. 295-324.
• KHOSRAVI, H. <italic>et al</italic> . Intelligent learning analytics dashboards: automated drill-down recommendations to support teacher data exploration. Journal of Learning Analytics , [s. l.], v. 8, n. 3, p. 133-154, Nov. 2021. https://doi.org/10.18608/jla.2021.7279
  » https://doi.org/10.18608/jla.2021.7279
• KONSTANTINIDIS, S. T. Internet of things in education. In : KONSTANTINIDIS, S.; BAMIDIS P. D.; ZORY, N. (Eds.). Digital innovations in healthcare education and training. Amsterdam: Elsevier, 2021. p. 61-86.
• KURILOVAS, E. Advanced machine learning approaches to personalise learning: learning analytics and decision making. Behaviour & Information Technology, [s. l.], v. 38, n. 4, p. 410-421, Apr. 2019. 10.1080/0144929X.2018.1539517
  » https://doi.org/10.1080/0144929X.2018.1539517
• LAJOIE, S.; AZEVEDO, R. Teaching and learning in technology-rich environments. In: ALEXANDER, P. A.; WINNE, P. H. (Eds.). Handbook of educational psychology. 2. ed. New York: Routledge, 2012. p. 803-823.
• LIU, D. Y.-T. et al. Data-driven personalization of student learning support in higher education. In: PEÑA-AYALA, A. (Ed.). Learning analytics: fundaments, applications, and trends. Cham: Springer International, 2017. p. 143-169.
• MARTINEZ-MALDONADO, R. et al . Physical learning analytics: a multimodal perspective. In: INTERNATIONAL CONFERENCE, 8 th , Sydney: ACM Press, 2018. Proceedings[...]. Available from: http://dl.acm.org/citation.cfm?doid=3170358.3170379 . Access in: 2019 June 6.
  » http://dl.acm.org/citation.cfm?doid=3170358.3170379
• MOTHUKURI, U. K. et al . Improvisation of learning experience using learning analytics in eLearning. In: 2017 5TH NATIONAL CONFERENCE ON E-LEARNING & E-LEARNING TECHNOLOGIES (ELELTECH), 5., Hyderabad, India: IEEE, 2017. Proceedings[…] . Available from: http://ieeexplore.ieee.org/document/8074995/ . Access in: 2019 June 7.
  » http://ieeexplore.ieee.org/document/8074995/
• MUSLIM, A. et al . The goal-Question-Indicator approach for personalized learning analytics. In: INTERNATIONAL CONFERENCE ON COMPUTER SUPPORTED EDUCATION, 9., CSEDU 2017. Porto. Proceedings[…] . Available from https://www.scopus.com/inward/record.uri?eid=2-s2.0-85023740180&partnerID=40&md5=c502f0b0510b6daacd769b919c086da0 . Access in: 2020 Aug 12.
  » https://www.scopus.com/inward/record.uri?eid=2-s2.0-85023740180&partnerID=40&md5=c502f0b0510b6daacd769b919c086da0
• NGUYEN, Q. et al. Examining the designs of computer-based assessment and its impact on student engagement, satisfaction, and pass rates. Computers in Human Behavior, [s. l.], v. 76, p. 703-714, Nov. 2017. https://doi.org/10.1016/j.chb.2017.03.028
  » https://doi.org/10.1016/j.chb.2017.03.028
• PAPPAS, I. O.; GIANNAKOS, M. N.; SAMPSON, D. G. Fuzzy set analysis as a means to understand users of 21st-century learning systems: the case of mobile learning and reflections on learning analytics research. Computers in Human Behavior, [s. l.], v. 92, p. 646-659, Mar. 2019. https://doi.org/10.1016/j.chb.2017.10.010
  » https://doi.org/10.1016/j.chb.2017.10.010
• RIENTIES, B.; CROSS, S.; ZDRAHAL, Z. Implementing a learning analytics intervention and e valuation framework: What works? <italic>In</italic> : Big data and learning analytics in higher education. [S. l.]: Springer, 2017. p. 147-166.
• ROMERO, L. et al. Supporting self-regulated learning and personalization using ePortfolios: a semantic approach based on learning paths. International Journal of Educational Technology in Higher Education, Cham, v. 16, n. 1, p. 16, Dec. 2019. https://doi.org/10.1186/s41239-019-0146-1
  » https://doi.org/10.1186/s41239-019-0146-1
• ROWE, E. et al. Assessing implicit science learning in digital games. Computers in Human Behavior, New York, v. 76, p. 617-630, Nov 2017. https://doi.org/10.1016/j.chb.2017.03.043
  » https://doi.org/10.1016/j.chb.2017.03.043
• SAIF, S. M. <italic>et al</italic> . Impact of ICT in Modernizing the global education industry to yield better academic outreach. Sustainability , [s. l.], v. 14, n. 11, 6884, June. 2022. https://doi.org/10.3390/su14116884
  » https://doi.org/10.3390/su14116884
• SAQR, M.; PEETERS, W.; VIBERG, O. The relational, co-temporal, contemporaneous, and longitudinal dynamics of self-regulation for academic writing. Research and Practice in Technology Enhanced Learning, Singapore, v. 16, n. 1, p. 29, Dec. 2021. https://doi.org/10.1186/s41039-021-00175-7
  » https://doi.org/10.1186/s41039-021-00175-7
• SHEMSHACK, A.; KINSHUK; SPECTOR, J. M. A comprehensive analysis of personalized learning components. Journal of Computers in Education , [s. l.], v. 8, n. 4, p. 485-503, Dec. 2021. https://doi.org/10.1007/s40692-021-00188-7
  » https://doi.org/10.1007/s40692-021-00188-7
• SHUTE, V. J. et al. Assessing key competencies within game environments. In: IFENTHALER, D.; PIRNAY-DUMMER, P.; SEEL, N. M. (Eds.). Computer-based diagnostics and systematic analysis of knowledge. Boston: Springer, 2010. p. 281-309.
• SIJING, L.; LAN, W. artificial intelligence education ethical problems and solutions In: INTERNATIONAL CONFERENCE ON COMPUTER SCIENCE & EDUCATION (ICCSE), 13.,. Colombo: IEEE, 2018. Available from: https://ieeexplore.ieee.org/document/8468773/ Access in: 2019 Apr. 10.
  » https://ieeexplore.ieee.org/document/8468773/
• SINGH, A. B.; MØRCH, A. I. An analysis of participants' experiences from the first international MOOC offered at the University of Oslo. Nordic Journal of Digital Literacy,[s. l.], v. 13, n. 1, p. 40-64, 12 mar. 2018. https://doi.org/10.18261/issn.1891-943x-2018-01-04
  » https://doi.org/10.18261/issn.1891-943x-2018-01-04
• SLADE, S.; PRINSLOO, P. Learning analytics: ethical issues and dilemmas. American Behavioral Scientist, Thousand Oaks, v. 57, n. 10, p. 1510-1529, Oct. 2013.
• SOLER COSTA, R. et al. Personalized and adaptive learning: educational practice and technological impact. Texto Livre: Linguagem e Tecnologia, Belo Horizonte, v. 14, n. 3, p. e33445, Sep. 2021. https://doi.org/10.35699/1983-3652.2021.33445
  » https://doi.org/10.35699/1983-3652.2021.33445
• SUNAR, A. S. et al . Personalisation in MOOCs: a critical literature review. In: ZVACEK, S. et al . (Eds.). Computer supported education . Cham: Springer International, 2016. p. 152-168.
• TEMPELAAR, D. et al. Student profiling in a dispositional learning analytics application using formative assessment. Computers in Human Behavior, v. 78, p. 408-420, jan. 2018.
• TERRAS, M. M. et al. The opportunities and challenges of serious games for people with an intellectual disability. British Journal of Educational Technology, London, v. 49, n. 4, p. 690-700, jul. 2018. https://doi.org/10.1111/bjet.12638
  » https://doi.org/10.1111/bjet.12638
• THOMAS, D.; BROWN, J. S. A new culture of learning: cultivating the imagination for a world of constant change. American Journal of Play, [s. l.], v. 219, n. 2, p. 121-123, 2011.
• THOMPSON, G.; COOK, I. The logic of data-sense: thinking through learning personalisation. Discourse, London, v. 38, n. 5, p. 740-754, Sep. 2017.
• VEKARIYA, V.; KULKARNI, G. R. Notice of violation of IEEE publication principles: hybrid recommender systems: survey and experiments. In: 2012 SECOND INTERNATIONAL CONFERENCE ON DIGITAL INFORMATION AND COMMUNICATION TECHNOLOGY AND IT’S APPLICATIONS (DICTAP), 2., 2012. Proceedings[…] Bangkok: IEEE, 2012. Available from: https://ieeexplore.ieee.org/document/6215409>. Access in: 2019 June 19.
  » https://ieeexplore.ieee.org/document/6215409>
• VERBERT, K. et al. Learning analytics dashboard applications. American Behavioral Scientist, [s. l.], v. 57, n. 10, p. 1500-1509, Oct. 2013. https://doi.org/10.1177/0002764213479363
  » https://doi.org/10.1177/0002764213479363
• VIVES-VARELA, T. et al. La autorregulación en el aprendizaje, la luz de un faro en el mar. Investigación en Educación Médica, México, v. 3, n. 9, p. 34-39, 2014.
• WAHEED, H. <italic>et al</italic> . Early prediction of learners at risk in self-paced education: a neural network approach. Expert Systems with Applications , v. 213, part A, p. 118868, Mar. 2023. https://doi.org/10.1016/j.eswa.2022.118868
  » https://doi.org/10.1016/j.eswa.2022.118868
• WILLIAMS, J. J.; KIM, J.; KEEGAN, B. Supporting instructors in collaborating with researchers using MOOClets. In: ACM CONFERENCE, 2., 2015. Proceedings[…] Vancouver: ACM Press, 2015. Available from: http://dl.acm.org/citation.cfm?doid=2724660.2728705 Access in 2019 June 7.
  » http://dl.acm.org/citation.cfm?doid=2724660.2728705
• WILLIAMS, P. Assessing collaborative learning: big data, analytics and university futures. Assessment & Evaluation in Higher Education, [s. l.], v. 42, n. 6, p. 978-989, Aug. 2017. https://doi.org/10.1080/02602938.2016.1216084
  » https://doi.org/10.1080/02602938.2016.1216084
• WORSLEY, M.; MARTINEZ-MALDONADO, R.; D’ANGELO, C. A new era in multimodal learning analytics: twelve core commitments to ground and grow MMLA. Journal of Learning Analytics, [s. l.], v. 8, n. 3, p. 10-27, Nov. 2021. https://doi.org/10.18608/jla.2021.7361
  » https://doi.org/10.18608/jla.2021.7361
• XING, W.; DU, D. Dropout prediction in MOOCs: using deep learning for personalized intervention. Journal of Educational Computing Research, [s. l.], v. 57, n. 3, p. 547-570, June 2019. https://doi.org/10.1177/0735633118757015
  » https://doi.org/10.1177/0735633118757015
• XU, W.; WU, Y.; OUYANG, F. Multimodal learning analytics of collaborative patterns during pair programming in higher education. International Journal of Educational Technology in Higher Education , [s. l.], v. 20, n. 1, p. 8, Feb. 2023. https://doi.org/10.1186/s41239-022-00377-z
  » https://doi.org/10.1186/s41239-022-00377-z
• YI, B. et al. Research on personalized learning model under informatization environment. In: INTERNATIONAL SYMPOSIUM ON EDUCATIONAL TECHNOLOGY (ISET), 2017, Hong Kong. Proceedings[…] Hong Kong: IEEE, June 2017. Available from: http://ieeexplore.ieee.org/document/8005386/ Access 2019 Apr. 10.
  » http://ieeexplore.ieee.org/document/8005386/
• ZHANG, J.-H. et al. What learning analytics tells us: group behavior analysis and individual learning diagnosis based on long-term and large-scale data. Educational Technology and Society, [s. l.], v. 21, n. 2, p. 245-258, 2018.

• Data: The set of data that supports the results of this review can be found directly in the sources stated in the article.

• Financing: No funding was received for the manuscript preparation and publishing nor its previous research processes.

Data availability

Data: The set of data that supports the results of this review can be found directly in the sources stated in the article.

Publication Dates

• Publication in this collection
  02 Feb 2024
• Date of issue
  Jan 2024

History

• Received
  09 Mar 2023
• Accepted
  10 Jan 2024