Language learning apps or games: an investigation utilizing the RETAIN model

Jogos ou apps para aprendizagem de línguas: uma investigação utilizando o modelo RETAIN

Abstract:

Combining games with mobile devices can promote learning opportunities at the learners' fingertips and enable ubiquitous learning experiences. As teachers increasingly assign games to reinforce language learning, it becomes essential to evaluate how effective these applications are in helping students learn the content or develop the skills that the games are reinforcing. This article examines two English language learning apps under the RETAIN model (GUNTER; KENNY; VICK, 2008). The findings indicate that although these apps offer some language learning opportunities, they do not present scenario-based quality or gameplay, among other elements, if they are to be considered games.

Keywords:
Educational Mobile Games; English Language Learning Applications; Evaluation of Educational Games; RETAIN Model.

Resumo:

A combinação de jogos e dispositivos móveis pode promover oportunidades de aprendizagem ao alcance dos aprendizes e permitir experiências de aprendizagem ubíquas. Uma vez que professores cada vez mais usam jogos para reforçar a aprendizagem de línguas, torna-se essencial avaliar o quão eficiente esses aplicativos são em ajudar os estudantes a aprender o conteúdo e desenvolver as habilidades que eles estão reforçando. Este estudo examina dois apps para a aprendizagem de língua inglesa utilizando o modelo RETAIN. Os resultados apontam que embora esses apps ofereçam algumas oportunidades de aprendizagem de línguas, eles não apresentam cenários ou gameplay entre outros elementos necessários para serem considerados jogos.

Palavras-chave:
Jogos Educacionais Móveis; Aplicativos para Aprendizagem de Línguas; Avaliação de Jogos Educacionais; Modelo RETAIN

1 Introduction

Games have existed for many years, but digital technology and the convergence of new media have enabled the social practice of playing to become commonplace on different devices, such as those connected to TVs (attached to video games), game consoles, and computers. With the explosion of new mobile devices - over one billion devices (KEMP, 2015), including smart phones, tablets and the like - one can now choose from a myriad of games and play them wherever and whenever they wish.

According to Scolari, Aguado, and Feijoo (2012), the upsurge of new mobile devices has created a new standard for mobile communications and media. The evolution of mobile applications and contents include, for instance, the transition from cell phones to multimedia devices, which integrate an array of functions and properties. Another evolution is the convergence of media that allows a vast range of contents, such as mobile user-generated content and mobile gaming. Hjorth (2011, p. 357) argues that "[...] just as the mobile media encompasses a range of affordances, practices and modes of co-presence, so too has mobile gaming grown with more possibilities."

Flurry Analytics conducted a survey of mobile app users, which indicated that gaming accounted for 43% of user time spent on mobile apps (CUTLER, 2012). Another survey in 2014, restricted to Android devices, because of its wider international reach, points to Android-based gaming as a global pastime, with gamers playing an average of 37 minutes a day (EUSTON, 2014). While many acknowledge the entertainment value of games, these devices are also being used as tools for learning.

Shelton, Satwicz, and Caswell (2013) point out that the educational value of play is acknowledged by both Piaget and Vygotsky. Based on Piaget (1932), the authors emphasize that the role of play is paramount to children's cognitive development. Along similar lines, Shelton, Satwicz, and Caswell (2013) revisit the work of Vygotsky (1962) and emphasize that the role of play is an essential element to the development of imagination in children and how both theories can contribute to new gaming technologies.

The interactive nature of games has led language teachers to integrate games such as crossword puzzles, word memory games, and vocabulary hangman in their practices as a vehicle to develop language skills. Learning activities for reinforcing language skills in the perspective of gamification often found in language teachers' repertoire include: Race to the Board, Word Jumble Race, Call my Bluff, and Simon Says. These activities and others present game-like features, such as competitions and races that are not only fun, but also educational. Recent studies worldwide (WANG; SHANG; BRIODY, 2011; SANTOS, 2012; MUBASLAT, 2012; BORGES, 2013) have recognized that students actively play games and use them to learn English, indicating that games must be taken into account to strengthen the student-school bond and provide students the opportunity to appropriate digital tools for learning that they already use in their day-to-day activities. Mobile games can be convenient, if we consider that one can learn in a fixed place or benefit from the mobility of the devices and Internet connections to play in networked environments at a distance.

With the language teacher perspective in mind and taking into consideration that mobile devices are the fastest growing technology platform in the developing world, this study includes the evaluation and discussion of two mobile educational games used for language learning. These language learning applications were selected because of their popularity for language learning. The RETAIN Model (GUNTER; KENNY; VICK, 2008) was used as the framework for analysis and discussion.

2 Literature Review

Motivations for language learning are changing, causing a need to re-examine and reflect on how learning opportunities should be designed for the next generation as well as for those returning to their language studies (KUKULSKA-HULME, 2015). With this in mind, a brief review of the literature that contributes to the discussions on designing and evaluating games in the language context is provided in the following sections.

2.1 Games Design

For years, educational games have been the focus of much discussion and debate. Since games have become an important social trend, many educators have confidence that, if used properly, they can become an important instructional tool. Well-developed mobile games can generate interactive, engaging, and immersive activities. Gunter and Gunter (2015) stated "[...] that while most games teach something, most of the positive outcomes of games used in instructional settings center on motivation, social skill-building, simulations, and changing attitudes, rather than on content-related skills. These outcomes are hard to measure, which has caused considerable confusion among game evaluators" (p. 273).

Gee (2013, p. 27) has for years questioned formal schooling, since it demands that students "[...] use their memories the way computers do, rather than the way humans do." Computers store data in a stable way; humans process information through dynamic association and from experiences. According to the researcher, the same holds true in most educational games available to our students: the heterogeneous and dynamic learning trajectory is not taken into account in the design process. Gunter, Kenny, and Vick (2008) contended there has been a recurrent failure to ground educational game design on well-established instructional theories. What happens more commonly is that educational game designers either infuse educational content into games or add game elements to educational content.

In the realm of digital games, mostly those for sale, Gee (2003) finds and points to learning principles which recur in good games that, despite the contextual informality in which they are used and the non-pedagogical purposes, are more conducive to effective learning than most educational games. Briefly, these principles indicate that learning takes place in a semiotic field in which multiple semiotic systems (images, words, symbols, actions, among others) are interwoven. Learning provides the redefinition of the players' identity, self, and trajectory towards achievement. From a situated meaning, knowledge is not only transferred, but also practiced as a social activity, under a cultural model, allowing feelings of affinity and belonging.

Educational games are viewed as those which are explicitly designed with the purpose of teaching specific subjects and, according to Torrente (2010), have the potential to enhance students' motivation. Some authors use the term 'instructional games' to refer to educational games. Hays (2010, p. 252) states that "an instructional game is specifically designed or modified to meet instructional objectives" and includes "rules, constraints, and activities that closely replicate the constraints of the real-world knowledge and skills that are being taught".

There are some Web-based and iOS scenario-oriented games that have been designed and developed for learning other languages. "Mentira"¹, designed by Holden and Sykes (2013), has been considered a good example of a game design, a language learning scenario-based game for students to acquire the Spanish language. "Mentira", a place-based, augmented reality game, engages learners in solving a murder mystery in a neighborhood in Albuquerque, New Mexico, through readings, videos, and images. Played on an iOS device, students interact with characters that come alive through the pictures, dialogs, and movies that they view and read on their mobile screen.

All phases of the design and production of an educational game require a great deal of planning and consideration to make sure that the media is linked to the content, the content is integrated and interlaced with game play, and learning is supported with well-planned hints and feedback (GUNTER; KENNY; VICK, 2008).

2.2 Teachers and Gaming

Globally, digital games in education are emerging (JOHNSON; ADAMS-BECKER; CUMMINS; ESTRADA, 2014; JOHNSON; ADAMS-BECKER; ESTRADA; FREEMAN, 2014). Teachers often turn to games and learning applications as tools for supplementing instruction, introducing content, and reinforcing skills. The National Foundation for Educational Research (NFER, 2009) conducted a survey with over 1600 British teachers in 2009 and, at that time, one-third of these respondents indicated the use of games in instruction. In comparison, 78% of 513 kindergarten through eighth grade teachers, mainly from the United States, use digital games in learning and 55% of their students play digital games weekly in class (TAKEUCHI; VAALA, 2014). The use of digital games is a common occurrence in students' everyday lives as well as in the classroom.

Teachers' reasons for selecting games vary. When the 513 teachers were asked to identify all of the factors that influence them in selecting games used in instruction, top reasons included the recommendation of other teachers (48%), assessment and analytical capabilities (43%), and personal experience and preferences (42%). Only 37% of respondents indicated that research about a particular game or evidence of the games impact contributed to the selection of a digital game used in learning. Yet 34% indicated that finding effective curriculum driven games was a barrier to the implementation of the games (TAKEUCHI; VAALA, 2014). Conversely, Ertzberger's (2009) study of 390 in-service and pre-service teachers indicated that games were chosen based on how the game supported the curriculum and technology in the classroom, and whether or not training was available. Finances and technological infrastructure presented barriers to game use.

With the increasing access to smartphones and mobile devices, and the rapid expansion of school-based 1:1² and BYOD³ programs, a need to investigate the efficacy of learning and game apps has emerged. The growth trend continues upward as indicated by the Horizon Report for Brazil and the United States (JOHNSON; ADAMS-BECKER; CUMMINS; ESTRADA, 2014) Because of the rapid adoption of mobile technologies, teachers, schools, and administrators alike have a newfound need to investigate the efficacy of potential learning applications and educational games that meet learning objectives.

In the United States, some school districts require teachers to complete multiple page documents evaluating games applications prior to using the apps. Once an app is approved for a classroom, school or school district, teachers can then incorporate the game or learning app into instruction; however, these evaluations may be based on a variety of factors not associated with the content and learning standards. Some teachers have indicated that school districts often do not consider the pedagogical implementation of the pre-approved apps, leaving some teachers to feel that these applications do not meet the students' academic needs.

Further, there are teachers that question the pedagogical effects and impacts of role playing and online games for learning. Chik's research of 34 in-service language teachers noted that teachers with minimal to no online gaming experience expressed apprehension and did not value online games for learning. The basis for their positions may be, as Chik (2011, p. 111) stated, that "language teachers, under the constraints of traditional classroom resources and curricula, might have their hands bound when it comes to envisioning gaming as learning tools.". Nevertheless, these same teachers recognized that students enjoyed playing digital games among learners.

Generally, teachers recognize the need to choose learning apps and games that meet curricular needs. The wide acceptance of digital games in education, the growth of mobile devices, and the need to support instruction, analyzing games and learning application with a research-based instrument grounded in pedagogy and theory is warranted. Knowing the strengths and weaknesses of a learning app or game allows teachers to best know how to use these types of digital resources. Researchers and practitioners in the educational field share the responsibility for better understanding the role of educational games in the learning process.

2.3 RETAIN Model

Aimed at analyzing and understanding games geared toward learning will help to identify those educational games that foster interaction, engagement, and immersion in a way that will benefit students in different educational settings. Gunter, Kenny, and Vick (2008) have contributed to the emerging field of educational game studies and responded to a challenge launched by Gee (2006) and others for researchers in different fields to rethink analytical tools being used to investigate gaming artifacts for educational purposes.

To meet the needs of the field, Gunter, Kenny, and Vick (2008) created a game evaluation and design rubric based on the RETAIN (Relevance, Embedding, Transfer, Adaptation, Immersion, Naturalization) model. The RETAIN rubric correlates instructional methods and learning theories that are closely aligned with generally accepted game design principles and theories, including Keller's ARCS Model and Gagne's Events of instruction (2010). Additionally, the rubric combines Piaget's thoughts on schema and Bloom's hierarchical structure for knowledge acquisition.

Each element of RETAIN provides a robust picture of a game's strengths and weaknesses. Relevance relates to the way the content of the game connects to the learners lives and the way instructional levels of the game connect to each other. Embedding cogitates the way the learning, fantasy, and game play are integrated in such a way that one does not overpower the other throughout the course of the game. Transfer reflects the way content knowledge is transferred to other novel scenarios or situations. Adaptation deepens the transfer in that assimilation and accommodation take place, in turn leading to the scaffolding of learning. Immersion indicates the games' fully immersive qualities. Finally, Naturalization, examines the automaticity of thought. The RETAIN model takes into account not only game elements, but also story/fantasy elements, which promotes a unique learning trajectory and inspires students to advance along a significant instructional path.

As a frame of reference, the RETAIN model has been used to evaluate serious computer-based games (ZHANG; FAN; XING, 2010), console games, and mobile learning apps (CAMPBELL; GUNTER; BRAGA, 2015), because it explains how well a game includes academic knowledge and transfer as a standalone approach to learning (GUNTER; KENNY; VICK, 2008; KENNY; GUNTER, 2011). Educators and computer science experts alike find the model to be useful for selecting serious games as instructional tools (PRINSLOO; JORDAAN, 2014). The RETAIN model weighs the strengths and weaknesses of a game for a teacher to become well informed as how to best use the game in learning.

3 Methodology

The study aimed at evaluating how effective mobile game applications were in aiding students to learn the required language skills and expected academic content. Specifically, this study presents an analysis of mobile language learning, using the RETAIN model developed by Gunter, Kenny, and Vick (2008). As researchers, the voice of the end user provided a basis for game selection. Since Android-based mobile devices have increased in popularity worldwide, a search of the Google Play Store was conducted to identify relevant apps.

3.1 The Search Method

Between December 2014 and February 2015, an ongoing search was conducted using the search feature in the Google Play Store through the auto selection of "Games" and "Top Free", "Top New Free", and "Trending" tabs. One of the prerequisites was that the games should be free of charge or based on a freemium model. Further, the language learning app needed to be popular among end users. The applications under these tabs are basically developed for entertainment purposes. For this reason, we narrowed down our search using a number of keywords, namely 'serious games', 'educational games', 'English', 'language learning', and 'learn language', with and without quotation marks.

Besides using these keywords, the individual game apps' pages always suggested other games in the "Similar apps" and "Users also installed" sections. The games in these sections were also considered so that the search could be as thorough as possible. Because of the number of games available, the following criteria for selecting the games analyzed in this study included:

1. a) Popularity - the games must have a high number of downloads: we located games that were downloaded 50, 10, and 5 million times;

2. b) User rating - users must have rated the games 4 stars or more;

3. c) Price - the games must be free of charge or based on a freemium model.

It is worth mentioning that both the store and the developers use the words 'game' and 'apps' interchangeably and it is sometimes difficult to identify the most adequate classification. Because of this limitation, we decided to depart from what the Play Store classified as games and to include in the analysis all the applications that fulfilled the selection criteria. The intention was to evaluate whether the applications met the evaluation criteria in the RETAIN model and presented characteristics of educational games, despite their being classified as such by the developers. Table 1 displays the list of games found.

Table 1:
Preliminary list of games

3.2 Narrowing down the list of games

Most of the identified games, as shown in the last column of Table 1, are classified within the categories of 'word' or 'puzzle'. Only four games go beyond vocabulary work and are classified under 'education', namely, "Duolingo: Learn Languages Free", "Learn English 6,000 Words", "Learn Languages - busuu", and "Babbel - Learn Languages". As established, the purpose of this study is to evaluate games that could facilitate players' language development. Considering that language acquisition involves more than just learning vocabulary items, only the games that incorporated communicative functions in contextualized situations of usage were considered, leaving four games under the broader category 'education'.

Despite being classified under the category 'education', the game "Learn English 6,000 Words" focuses on vocabulary work, including various gamified activities with thousands of words organized in topics. As for "Babbel - Learn Languages", although the game incorporates functional language, only the first lesson in each topic is available for free and most of the activities require a subscription; as a result, the game does not really meet the price criteria. After completing the few lessons that are free, it was determined that "Babbel - Learn Languages" is not a game, but rather a learning app, as it is composed of a set of language lessons that do not present any kind of gamification. With these two exclusions, the preliminary list of games was finally narrowed down to "Duolingo: Learn Languages Free" and "Learn Languages - busuu".

3.3 Description of the games

Learn Languages - Busuu⁴ (henceforth Busuu) offers different learning opportunities, both individual and collaborative. The goal of this mobile learning app is to offer opportunities to learn the most important 150 topics and 3,000 words. The game menu displays the gamer's photo or image and includes five sections: Courses, Help Others, My Exercises, Preferences, and Contact us.

The first section on the list - Courses - presents lessons based on everyday topics, which unfold into language functions, grammar, and writing. Vocabulary tasks, dialogs, and quizzes are also part of the lesson and are meant to be done individually. The writing tasks can be shared with native speakers and other gamers.

The section entitled Help Others provides collaborative learning as gamers have a chance to share their writings with other gamers in addition to giving and receiving feedback. My exercises serves as a portfolio for writing tasks. Preferences, in turn, is a space where account information and user profile can be found. In Contact us, gamers can interact with the game's developers.

The game has five proficiency levels - from beginner to upper intermediate - each with two sets of lessons. Most activities in each lesson are available for free. Others are locked and require payment. An English-for-travel course is also available, but, again, most lessons require payment.

Duolingo: Learn Languages Free⁵ (henceforth Duolingo) provides "free language education for the world" where learning is gamified. Players advance in the game by unlocking bite-sized activities as they try to finish each lesson without losing their hearts, the game's progress indicator. The languages available depend on the players' mother tongue: if Portuguese, they can choose from English, Spanish, and French (and soon, German), compared with more than 10 languages if the players speak English. During the setup, students must "pick a daily goal" to plan their studies from a list that includes: 'Casual', 5 minutes a day; 'Regular', 10 minutes a day; 'Serious', 15 minutes a day; and 'Insane', 20 minutes a day. The game uses this setting to track the progress and send occasional reminders if players stay out for too long. Players finish the initial settings, defining whether they are beginners and will start at the 'Basics' level or they will take the 'Placement Test'. When players start a level, like 'Basics 1', they know how many lessons it has and what the main content is.

The game's content is organized in topics, which include Basics, Food, Objects, Adjectives, Questions, Present, etc., which are further divided into lessons that work with grammar and vocabulary, both as isolated language items and as parts of phrases and sentences.

Each lesson offers a set of activities, including: unscrambling a sentence from oral input, translating a phrase with scrambled prompts, translating a sentence from written input, translating a sentence from oral input, translating a sentence from both written and oral input, typing what you hear from slow or natural speed input, choosing the picture and word for a vocabulary item, choosing all the right translations from a list, and repeating a sentence from both written and oral input. These lessons must be done in a pre-established sequence and players must finish one to have access to the next. However, they can skip any individual topic if they think they already know the content enough by clicking on 'Test Out' and taking a specific placement test. The game has a voice recognition system that allows players to say a sentence and receive feedback on how good their pronunciation is.

The gamification elements include, for example: players lose 'hearts' when they give wrong answers; they are encouraged to do extra exercises to review and gain more hearts; players gain both hearts and experience points as they give right answers and complete the lessons; players can track their progress with Duo's, the game's owl symbol, regular feedbacks; players can gain 'lingots' if they do not make any mistakes, with which they can buy more hearts, extra lessons, and outfits for the owl, personalizing the experience; players are encouraged to add friends and start competing.

3.4 Evaluation procedures

The researchers individually used the RETAIN model rubric to make a complete evaluation of the games. Once all participants had their personal notes and scores completed for both games, we discussed the notes and scores in order to reach a unified evaluation. The integrated evaluation is presented in the following section.

4 Findings and discussion

The elements of the rubric will be used one by one to analyze for both games so that the contrast can help us to have a better understanding of how to use the element in the game's analysis and a better view of the isolated evaluations. We will start by highlighting in tables the key ideas that summarize the conceptual design for each level of each element as presented in the rubric by the authors, and then proceed to the actual evaluation of the games.

Figure 1:
RETAIN rubric and rating based on the element Relevance Copyright 2005-2016. Reprinted with permission from the authors, Gunter, Kenny, & Vick. All rights reserved

Busuu: The gameplay centers on language functions commonly found in everyday life and presents pedagogic elements that promote language learning opportunities. It lacks imaginary or fantasy context and basically revolves around everyday living situations. Although the lack of fantasy and stories may be problematic, the contextualization of the learning elements creates stimulus for the player to stay in the game and seek any other topic that (s)he desires to explore. Although the pedagogic elements provided after vocabulary presentation are targeted to defined learning objectives, the learning app does not demonstrate a scenario based or role playing immersive quality. Finally, the characters were unfamiliar to other players.

Score = weight (1) x level (0) = 0 out of 3

Duolingo: The gameplay centers on vocabulary and grammar tasks organized in topics. The pedagogic elements are defined as there are clearly stated goals in each set of activities. Unreal and decontextualized sentences like "the dogs drink wine", "we are reading the horses", and "we eat apples" are problematic as they are irrelevant to players' lives or the world around them. In addition, most activities are repeated throughout the game and do not exactly go much further than the basic level. For this reason, the cognitive challenges are limited, as is the game's relevance to students' lives. The highly frequent vocabulary items and grammar points presented, as well as the possibility of competing with friends, may create an interest among players. Like Busuu, Duolingo does not use characters and themes that the players are familiar with and lacks contextualization and fantasy.

Score = weight (1) x level (0) = 0 out of 3

Figure 2:
RETAIN rubric and rating based on the element Embedding Copyright 2005-2016. Reprinted with permission from the authors, Gunter, Kenny, & Vick. All rights reserved

Busuu: Although Busuu does not offer story/fantasy, it promotes contextualized learning and uses multimodal texts (picture, written and oral texts) to facilitate learning. Both the multimodal texts and activities have been created around themes and situations of real life, which are articulated around vocabulary and communicative functions, including but not limited to: feelings, appearances, and talking on the phone. The game promotes collaborative writing based on the themes. The writing activity in the unit "Appearances" encourages students to use language content to briefly describe themselves. The writing activities are subject to feedback from native speakers and Busuu peers which offers intellectual challenges of sufficient level to keep users interested in completing the lesson and gaining "berries" (rewards).

Score = weight (3) x level (0) = 0 out of 9

Duolingo: The learning application has no interactive focus on the emotional, psychological, physical, and intellectual levels, as most tasks require rote memorization. It allows for experiences with contexts related to English language teaching curriculum, but in a more structural way. Vocabulary and grammar are the content, and there are no communicative functions. Vocabulary and grammar are presented in a decontextualized manner - sentences isolated from real-life situations. Intellectual challenges are somehow presented to players of sufficient level to keep them interested in completing the game. Players need to complete a lesson to open a new one, which may be considered a form of intellectual challenge. It is important to mention that there is no fantasy context and, as a consequence, no story for the content to be embedded in.

Score = weight (3) x level (0) = 0 out of 9

Figure 3:
RETAIN rubric and rating based on the element Transfer Copyright 2005-2016. Reprinted with permission from the authors, Gunter, Kenny, & Vick. All rights reserved

Due to the current limits of technology in learning applications, 3D cues and interactive animations were not considered in the rating.

Busuu: The progression happens easily and facilitates the transfer of knowledge during pedagogic activities. Instructional elements are introduced in a hierarchical manner so that knowledge acquired can be transferred to other moments of the lesson. For example: first, isolated lexical items are presented, next they are used in communicative function, and then a writing activity is proposed in such a way that the activities are presented in a hierarchical manner. In the unit, "Food and Drink" the objective is to create a shopping list, first, players learn isolated items about the theme, then they listen to a dialog about making a shopping list, and finally they click and drag items into the shopping list as a writing activity. The writing task can be considered an opportunity for knowledge transfer. To accomplish this task, learners have to use the language developed in previous vocabulary and grammar tasks. The game also offers levels of challenge within the dialogs since the language becomes more complex as the game evolves. However, our analysis shows that little has been done to integrate the lessons throughout the learning application.

Score = weight (5) x level (1) = 5 out of 15

Duolingo: Language level progression is demonstrated throughout Duolingo. The levels start with basic vocabulary that is progressively associated with grammar topics. For example, the very first topic, "Basic 1", starts with isolated words like "man", "woman", "boy", and "girl" presented with a flashcard activity, then it includes the article "a" in the following activity, and finally moves on to work with these words/phrases in sentences like "I am a man" in translation activities. It contains popups indicating right (green flag) or wrong (red flag) after each activity facilitating the transfer of knowledge during pedagogic events. However, only a few include language feedback like "You used the indefinite article 'a' instead of the definite article 'the'" presented on a red flag. Players are able to progress through the levels easily, since completing a lesson is required to move on to the next level. Once instructional elements are introduced in a hierarchical manner and repeated throughout the game, players can progress smoothly and transfer the knowledge gained to other situations. Content is usually repeated throughout the set of activities, allowing students to use the same vocabulary or grammar topic in different ways - repetition may help students notice what the core content really is and be more aware of it. Duolingo lacks authentic real life experiences that reward meaningful "post-event" knowledge acquisition, and it does not contain "after action reviews" that offer players an opportunity to teach others what they have learned.

Score = weight (5) x level (2) = 10 out of 15

Figure 4:
RETAIN rubric based on the element Adaptation Copyright 2005-2016. Reprinted with permission from the authors, Gunter, Kenny, & Vick. All rights reserved

Busuu: Busuu builds upon the users' existing cognitive structure because of the didactic sequence and its increase in complexity. The units are developed in levels based on the Common European Framework of Reference for Languages: learning, teaching, assessment unfold in such a way that that new structures are constructed on the previously presented grammar and vocabulary. Content is sequenced so that previous vocabulary schema is transferred to the next context/text given. For example, the lesson on "Feelings" in Beginners A1: player-learners learn the vocabulary they need to understand and complete the tasks related to a dialog in which two people express their feelings. Another point worth mentioning is that a certain degree of adaptation also happens when learners rely on existing knowledge of language functions in his or her native language and identify and relate lexical items in their native language and in English when completing the proposed translation tasks. The writing activities can be considered examples of active participatory processes, as players are required to go beyond what was given and transfer what they have learned to the production task. The possibility of giving and receiving feedback in the writing activity promotes the construction of new ideas, that is, players must not only know the language, but also think about the language when they give feedback to peers. In this sense, learning is seen as an active, participatory process.

Score = weight (4) x level (2) = 8 out of 12

Duolingo: Duolingo builds upon the player's existing cognitive structure, as the activities strongly rely on the player's native language for the translation exercises. There is a repetition of same task structure - translation, dictation, multiple choice, unscrambling, and matching exercises - in which players can identify old schema and transfer into new ones. However, the repetition and limited range of content do not provide enough environment for the discovery of new concepts or the construction of new ideas - the experiences the activities foster are simply not rich and varied enough. Instruction is not designed to encourage users to go beyond given information, and they take a more passive role in constructing new ideas. Players may associate new information with previous learning, but the design does not guarantee that.

Score = weight (4) x level (1) = 4 out of 12

Figure 5:
RETAIN rubric and rating based on the element Immersion Copyright 2005-2016. Reprinted with permission from the authors, Gunter, Kenny, & Vick. All rights reserved

Busuu: Players develop tasks related to thematic topics, but they do not have an opportunity for immersion in gameplay due to a lack of game elements, except for the gathering of Busuu berries. It provides no progressive, formative feedback during each unit: the player will receive automatic feedback of right (green check) or wrong (red X) for the multiple choice and completion activities in each unit. It presents little opportunity for reciprocal action and active participation in the writing tasks; moreover, this participation does not fit gameplay. Like other elements of the lesson/play, it is only involved with the didactic focus. The game lacks the element of play outside the pedagogic-related mechanics of the tasks. It does not create fantasy for players to stay in the action.

Score = weight (2) x level (0) = 0 out of 6

Duolingo: Elements of play are present throughout the units. While players are answering proposed tasks, they can visualize a progress bar indicating a percentage that may increase or decrease depending on the correct and wrong answers. These elements may foster a sense of challenge and the possibility of rewards. The feedback received is progressive and formative since it does not offer only a right or wrong response. For example, in a Simple Present exercise the players are asked to translate the phrase "She works" from their mother tongue into English. If we write "She work", the game shows a red flag with the information "'work' becomes 'works' when using the third person singular. She works." Besides this feedback from the game, the red flag has a speech bubble icon that leads to comments where players help each other solve their doubts. However, it lacks in offering opportunities for reciprocal action, shared responsibility, active participation, and belief creation. Although it works with heart icons that can be lost during each lesson, there is no clear emotional involvement in its content, since it is not very clear how losing hearts interferes in the game as a whole. There is actually cognitive involvement at stake. Besides keeping the "hearts", right answers give students "lingots" at the end of a lesson. With these "lingots", students can buy Duo, the game's owl symbol, a new outfit (formal or sports) or bonus units (idioms and sayings, or pickup lines). Though these elements could contribute to creating a sense of investment in the context of the game, they do not seem interesting enough and do not add anything special to it.

Score = weight (2) x level (1) = 2 out of 6

Figure 6
RETAIN rubric and rating based on the element Naturalization Copyright 2005-2016. Reprinted with permission from the authors, Gunter, Kenny, & Vick. All rights reserved

Busuu: Sequenced and repeated play is encouraged but the game centers on similar kinds of tasks. Replay is not encouraged at the end of the lesson, since users need to do all of the activities over again, and the design is not organized in such a way that users know clearly where they went wrong. The use of combined media - image, text, and sound - foster the improvement of the speed of cognitive response, automaticity, and visual processing. However, it only encourages the processes of synthesis, critical judgment, content awareness, and spontaneity to use the knowledge acquired in the review questionnaires when the system identifies errors in the process and provides the same question. Little variation may discourage players from staying active. This is problematic, because no novelty or curiosity is added to the practice.

Score = weight (6) x level (1) = 6 out of 18

Duolingo: Replay is always encouraged to assist in retention and to remediate any shortcomings of the process. For example, whenever the player completes the exercises in a topic, say "Verbs: Future", a final lesson is created with extra exercises on the words the players had problems with during the regular set of lessons. The types of activities are somewhat varied, but they are repeated throughout the game, like translation, unscrambling, and multiple-choice exercises. This repetition may cause users to improve their speed of cognitive response, automaticity, and visual processing. On the other hand, the game does not really encourage the processes of synthesis, critical judgment, content awareness, and spontaneity to use the knowledge acquired, as players have pretty much the same from the beginning to the end. As the game works with decontextualized phrases and sentences, it is not possible to infer that the users will be able to use the knowledge efficiently in real situations of language use.

Score = weight (6) x level (1) = 6 out of 18

Table 2:
Table of integrated analysis

Mobile technologies play an important role in language learning in formal and informal settings. Students tend to be motivated to learn by game play (GEE, 2012; 2013), both in the language learning classroom (KUKULSKA-HULME, 2015; HOLDEN; SYKES, 2013) and for personal self-improvement. While the intent of the research was to evaluate educational language learning mobile game applications, using those chosen in this study, these learning applications do not reflect the qualities of a serious game as scored using the RETAIN model. The mobile applications Busuu and Duolingo, contained some game elements, but these were more drill and practice, flash-card oriented language learning activities. These applications lacked the scenario, story/fantasy-based, problem-based aspects that are often found in educational games (GUNTER; KENNY; VICK, 2008; GUNTER; GUNTER, 2015).

Although Busuu and Duolingo do not have a story or fantasy and are not immersive in nature to attract players/learners, 5 to 10 million people have downloaded them, and many consider the apps useful or meeting their expectations based on the users' game ratings. One of the hypotheses for the volume of downloads may be the large demand for learning additional language, and the platform of mobile device makes language learning apps convenient (time and place) and personalized. Another hypothesis may be related to the users' concept of how language is acquired by practicing vocabulary.

Our findings indicate that although as a stand-alone educational game Busuu and Duolingo scored low on the RETAIN model, these language learning applications do in fact have a place in learning environments. Their contributions to introducing and practicing vocabulary and grammar contribute to language learning. When integrated in a classroom environment, the teacher can use these learning apps as supplemental language practice. Busuu and Duolingo like flashcards, sorting and classifying words activities, and fill in the blank worksheet support language learning but not in the same way as a serious game. Educational mobile game apps that are well designed have the potential to engross the player/learner in the content in an authentic, immersive, and engaging manner.

In this light, a good educational game would be one in which the fantasy/story content and gaming elements are not merely peripheral, but rather central features that organize and embed the academic content in a seamless manner. Players should become so involved both mentally and emotionally in the gameplay that they would not be able to tell content from gaming and would completely engage in the ongoing experience of the game. Interactive animations and 3D elements would be hallmarks of this type of experience. Problem-based learning scenarios, role playing situations, augmented reality, and simulations should be considered in mobile learning applications.

According to the RETAIN model, Busuu and Duolingo strengths included Transfer, Adaptation, and Naturalization. Learning levels are sequenced and integrated, allowing learners to use the knowledge developed in one level progressively as they move on to higher levels. Busuu has a social aspect in that users can practice what they have learned through questions and answers and then receive peer feedback in the writing activities. In both apps, learners are told what to do in the activities by structured instructions, while in a real games, players usually discover the rules inductively as they move on in the story; instructions may be present, but they do not have to be explicit. Moreover, the challenges are organized not only to allow players to apply previous content knowledge (vocabulary, grammar) in other areas, but also previous communicative competencies (functions) and skills (communication, cognitive, metacognitive). If the game incorporates activities that make learners do more than is developed within the game, that is, move from controlled activities to free tasks, they will be promoting these learners' autonomy.

A limitation of the evaluation was found when using learning applications that did not have the characteristics of a serious game. The RETAIN model was designed for the evaluation of educational games, not mobile learning applications with gaming elements. That having been said, these mobile learning apps, if combined with other instructional tools, could still receive a passing score. If the teacher integrated them with other content specific curriculum tools, and various instructional strategies that met all the elements of the RETAIN model, the classroom and the tools could create a better learning experience. Further limitations included not purchasing the full application and only using the free aspects of the language learning applications for evaluation. While the full version of the language learning applications were not purchased, evidence of any major differences of the freemium to purchased applications were minimal and would have not effected the evaluative scores.

5 Final remarks

Language learning apps with game elements, like Bussu and Duolingo, offer learners opportunities to develop skills that can be transferred to other contexts, be they the language classroom or other real world social practices. Gee (2012, p. xiv) defends that the main advantage of games for language learning is to situate meaning: "Games associate words with images, actions, goals, and dialog, not just with definitions of other words." In this view, the analyzed learning apps show potential value in connecting vocabulary and images as a starting point for language learning. Busuu's dialog feature may build capacity and facilitate transfer by scaffolding levels of challenge.

Based on the learning apps reviewed in this study and those available for mobile learning, there is an evident need for more language learning mobile applications and language learning educational games for mobile platforms. Furthermore, future language learning game designers could consider designing activities that are immersive and authentic, offer role playing or fantasy, and provide a context for discovery and decision-making. The activities included would have a communicative feature and alternative paths that allow learners of varying levels to travel along paths geared towards the transfer of knowledge. Adding these types of elements and characteristics to learning applications would move learning applications with game elements to educational games that could potentially engage learners in an immersive manner.

Finally, teachers have an array of learning supports to choose from to teach and reinforce academic content, including educational games and mobile learning applications. Knowing which tool to choose requires knowing the desired outcome and purposeful evaluation of the learning support. In this light, the RETAIN model provides a meaningful way for teachers to evaluate educational games for language learning.

References:

• BORGES, Anderson Pimentel. Jogo digital para reconhecimento de palavras: Análise comparativa entre as versões com instruções implícitas e explícitas. 2013. Doctoral Dissertation (Ph.D. in Linguistics) - Faculdade de Letras, Universidade Federal de Minas Gerais, Belo Horizonte, Minas Gerais. 2013.
• CAMPBELL, Laurie O.; GUNTER, Glenda A.; BRAGA, Junia. Utilizing the Retain Model to Evaluate Mobile Learning Applications. 2015. In: PROCEEDINGS OF SOCIETY FOR INFORMATION TECHNOLOGY & TEACHER EDUCATION INTERNATIONAL CONFERENCE 2015. Chesapeake, VA: Association for the Advancement of Computing in Education (AACE), 2015. p. 670-674.
• CHIK, Alice. Digital gameplay for autonomous foreign language learning: gamers' and language teachers' perspectives. In: REINDERS, Hayo (Ed.). Digital games in language learning and teaching. New York: Palgrave Macmillan, 2012. p. 95-114.
• CUTLER, Kim-Mai. Time spent in mobile apps is starting to challenge television, Flurry says. December 5, 2012. Retrieved from: < Retrieved from: http://techcrunch.com/2012/12/05/flurry-mobile-apps-television/ >. Accessed on: April 30, 2015.
  » http://techcrunch.com/2012/12/05/flurry-mobile-apps-television/
• EUSTON, Jarah. Gaming: The Lingua Franca of mobile. September 18, 2014. Retrieved from: < Retrieved from: http://www.flurry.com/blog/flurry-insights/gaming-lingua-franca-mobile#.VSHM-PnF8bg >. Accessed on: April 30, 2015.
  » http://www.flurry.com/blog/flurry-insights/gaming-lingua-franca-mobile#.VSHM-PnF8bg
• ERTZBERGER, J. An Exploration of Factors Affecting Teachers' Use of Video Games as Instructional Tools. 2009. In: GIBSON, I. et al. (Eds.). Proceedings of Society for Information Technology & Teacher Education International Conference 2009. Chesapeake, VA: Association for the Advancement of Computing in Education (AACE), 2009. p. 1825-1831.
• GEE, James Paul. What video games have to teach us about learning and literacy. New York: Palgrave Macmillan , 2003.
• GEE, James Paul. Why game studies now? Video Games: A New Art Form. Games & Culture, Sage Publications, vol. 1, n. 1, p. 58-61, January 2006.
• GEE, James Paul. Forward. In: REINDERS, Hayo (Ed.). Digital games in language learning and teaching. New York: Palgrave Macmillan , 2012. p. xii-xiv.
• GEE, James Paul. The anti-education era: creating smarter students through digital learning. New York: Palgrave Macmillan , 2013.
• GUNTER, Glenda A.; GUNTER, Randolph E. Teachers Discovering Computers: Integrating Technology in a Changing World. 8 ed. Boston, MA: Cengage Learning, 2015.
• GUNTER, Glenda A.; KENNY, Robert F.; VICK, Erik H. Taking educational games seriously: using the RETAIN model to design endogenous fantasy into standalone educational games. Education Tech Research Dev, vol. 56, n. 5-6, p. 511-537, 2008.
• HAYS, Robert T. Making games more effective in the classroom. In: HIRUMI, Atsusi (Ed.). Playing games in school: video games and simulations for primary and secondary education. ISTE, 2010. p. 249-276.
• HJORTH, Larissa. Mobile game cultures: The place of urban mobile gaming. Convergence: The International Journal of Research into New Media Technologies, vol. 17, n. 4, p. 357-371, November 2011.
• HOLDEN, Chris; SYKES, Julie. Leveraging Mobile Games for Place-Based Language Learning. In: FELICIA, Patrick. (Ed.). Developments in current game-based learning design and deployment. Hershey PA: IGI Global, 2013. p. 27-45.
• JOHNSON, L.; ADAMS-BECKER, S.; CUMMINS, M.; ESTRADA, V. 2014 NMC Technology Outlook for Brazilian Universities: A Horizon Project Regional Report. Austin, Texas: The New Media Consortium, 2014.
• JOHNSON, L.; ADAMS-BECKER, S.; ESTRADA, V.; FREEMAN, A. NMC Horizon Report: 2014, K-12 Edition. Austin, Texas: The New Media Consortium , 2014.
• KELLER, John. Motivational Design for Learning and Performance: The ARCS Model Approach. New York: Springer, 2010.
• KEMP, Simon. Digital, Social & Mobile Worldwide in 2015. January 21, 2015. We are social [Blog]. Retrieved from: < Retrieved from: http://wearesocial.net/blog/2015/01/digital-social-mobile-worldwide-2015 >. Accessed on: May 21, 2015.
  » http://wearesocial.net/blog/2015/01/digital-social-mobile-worldwide-2015
• KENNY, Robert F.; GUNTER, Glenda A. Factors affecting adoption of video games in the Classroom. Journal of Interactive Learning Research, Chesapeake, VA: AACE, vol. 22, n. 2, p. 259-276, 2011.
• KUKULSKA-HULME, Agnes. Language as a Bridge Connecting Formal and Informal Language Learning Through Mobile Devices. In: WONG, Lung-Hsiang; MILRAD, Marcelo; SPECHT, Marcus. (Eds.). Seamless Learning in the Age of Mobile Connectivity. Singapore: Springer Science+Business Media, 2015. p. 281-294
• MUBASLAT, Mania Moayach. The effect of using educational games on the students' achievement in English language for the primary stage. 2012. Retrieved from: < Retrieved from: http://files.eric.ed.gov/fulltext/ED529467.pdf >. Accessed on: May 28, 2015.
  » http://files.eric.ed.gov/fulltext/ED529467.pdf
• NFER Teacher Voice Omnibus Survey. Using computer games in the classroom. 2009. Retrieved from: < Retrieved from: http://www.nfer.ac.uk/nfer/what-we-offer/teacher-voice/PDFs/futurelab.pdf >. Accessed on: May 23, 2015.
  » http://www.nfer.ac.uk/nfer/what-we-offer/teacher-voice/PDFs/futurelab.pdf
• PIAGET, Jean. The moral judgement of the child. Illinois: The Free Press Glencoe, 1932.
• PRINSLOO, J. W.; JORDAN, D. B. Selecting Serious Games for the Computer Science Class. Mediterranean Journal of Social Sciences, vol. 5, n. 21, p. 391-298, 2014.
• SANTOS, Vinicius R. F. Os jogos MMORPG como auxiliares no processo de aquisição de língua inglesa. 2011. 121 f. Masters Thesis (Masters in Linguistics) - Faculdade de Letras, Universidade Federal de Minas Gerais, Belo Horizonte, Minas Gerais.
• SCOLARI, Carlos A.; AGUADO, Juan M.; FEIJÓO, Claudio. Mobile Media: Towards a Definition and Taxonomy of Contents and Applications. International Journal of Interactive Mobile Technologies, vol. 6, n. 2, 2012.
• SHELTON, Brett E.; SATWICZ, Tom; CASWELL, Tom. Historical perspectives on games and education from the learning sciences. In: FELICIA, Patrick. (Ed.). Developments in current game-based learning design and deployment. Hershey PA: IGI Global , 2013. p. 339-364.
• TAKEUCHI, Lory M.; VAALA, Sarah. Level up learning: A national survey on teaching with digital games. New York: The Joan Ganz Cooney Center at Sesame Workshop, 2014. Retrieved from: < Retrieved from: http://www.joanganzcooneycenter.org/wp-content/uploads/2014/10/jgcc_leveluplearning_final.pdf >. Accessed on: May, 21, 2015.
  » http://www.joanganzcooneycenter.org/wp-content/uploads/2014/10/jgcc_leveluplearning_final.pdf
• TORRENTE, J. <e-Adventure>: Introducing educational games in the learning process. Education Engineering (EDUCON), 2010. p. 1121-1126.
• VYGOTSKY, L. S. Thought and Language. Cambridge, MA: MIT Press, 1962.
• WANG, Yin-Jian; SHANG, Hui-Fang; BRIODY, Paul. Investigating the impact of using games in teaching children English. International Journal of Learning and Development, v. 1, n. 1, p. 127-141, 2011. Retrieved from: < Retrieved from: http://www.macrothink.org/journal/index.php/ijld/article/viewFile/1118/874 >. Accessed on: May 24, 2015.
  » http://www.macrothink.org/journal/index.php/ijld/article/viewFile/1118/874
• ZHANG, H. F.; FAN, X. Y.; XING, H. F. Research on the Design and Evaluation of Educational Games Based on the RETAIN Model. 2010. In: 2010. THIRD INTERNATIONAL SYMPOSIUM ON KNOWLEDGE ACQUISITION AND MODELING, 2010. p. 375-378.

• 1
  http://www.mentira.org/
• 2
  1:1 - One to One: Every student in the class has his/her own school-provided device (e.g. computer, laptop, tablet, or smartphone).
• 3
  BYOD - Bring Your Own Device: Students bring their own personal device to school to use for learning activities (e.g. any iOS or Android based device).
• 4
  http://www.busuu.com
• 5
  http://www.duolingo.com

Publication Dates

• Publication in this collection
  05 Apr 2016
• Date of issue
  Apr-Jun 2016

History

• Received
  20 Feb 2015
• Accepted
  25 July 2015