Virtual reality in the rehabilitation of patients with stroke: an integrative review

Realidade virtual na reabilitação de pacientes após acidente vascular cerebral: uma revisão integrativa

ABSTRACT

Objective:

To describe the intervention protocols to using commercial video games as virtual reality (VR) in rehabilitation of patients with stroke.

Methods:

Integrative review using the descriptors “rehabilitation”, “virtual reality exposure therapy” and “videogames” in the LILACS and PUBMED databases. Articles published from 2011 to 2018 were selected.

Results:

We found 1,396 articles, 1,383 were excluded and 13 were selected. Most of the articles were randomized clinical trials published in 2014 or later. The sample size varied from 5–47 adults, or adults and elders, with chronic stroke. The Nintendo Wii^® was the most used video game system. The intervention happened two or three times a week, each session lasting from 30 to 60 minutes, over 2–12 weeks. Balance, upper limb motor functions, quality of life and daily living activities were the most common evaluated outcomes. The Fugl-Meyer Assessment, Berg Balance Scale, Timed Up and Go test, Barthel Scale and SF-36 were the most common outcome measurement tools.

Conclusions:

The studies indicated improvement in dynamic balance, upper limb motor function and quality of life after rehabilitation using VR. The VR was more effective than conventional treatments for the outcome of dynamic balance. Two studies did not find any changes in static balance and daily living activities. Physical aspects and quality of life were the outcomes most evaluated by the researchers; as were the population with chronic strokes and protocols of long duration and low intensity. Few studies targeted immediate VR effects, performance in daily living activities and social participation.

Keywords:
Rehabilitation; virtual reality exposure therapy; video games; stroke

RESUMO

Objetivo:

Descrever os protocolos de intervenção utilizando sistemas de vídeo game comerciais como realidade virtual (RV) na reabilitação de pacientes após AVE.

Método:

Revisão integrativa usando os descritores “reabilitação”, “terapia de exposição a realidade virtual” e “videogames”, nas bases de dados LILACS e PUBMED, e artigos publicados entre 2011 e 2018.

Resultados:

Foram encontrados 1.396, excluídos 1.383 e selecionados 13 artigos. A maioria era ensaio clínico aleatorizado publicados a partir de 2014. O tamanho da amostra variou de 5 a 47 participantes adultos ou adultos e idosos com AVE crônico. O Nintendo Wii^® foi o videogame mais empregado. A intervenção constava de duas ou três sessões semanais, com duração de 30 ou 60 minutos por um período de duas a 12 semanas. Equilíbrio, função motora de membro superior, qualidade de vida e atividade de vida diária foram os principais desfechos. Empregou-se principalmente Escala Fugl-Meyer, Escala de Equilíbrio de Berg, teste Timed Up and Go, Índice de Barthel e SF-36 como medidas de desfecho.

Conclusões:

Os estudos indicaram melhora do equilíbrio dinâmico, função motora de membro superior e qualidade de vida após reabilitação usando RV. A RV se mostrou mais eficaz que tratamentos convencionais para o desfecho equilíbrio dinâmico. Dois estudos não encontraram mudanças no equilíbrio estático e atividade de vida diária. Observou-se maior foco em desfechos relacionados a aspectos físicos e qualidade de vida, em população com AVE crônico e protocolo com maior duração e menor intensidade. Poucos estudos voltados para efeitos imediatos da RV, desempenho em atividades de vida diária e participação social.

Palavras-chave:
Reabilitação; terapia de exposição à realidade virtual; jogos de vídeo; acidente vascular cerebral

Rehabilitation of a patient is a process that involves the knowledge of the professional regarding the health condition and the repercussion it has on the life of the individual, as well as a broad scientific knowledge about the functioning of the human body, so that consistent decisions may be made¹. In this process, assessing the situation, planning and choosing the best available evidences to perform clinical tasks is paramount to systematize decisions and optimize results².

Many rehabilitation techniques described in the literature may help guide the intervention of professionals, such as physical training, kinesitherapy, robotic therapy, hydrotherapy, music therapy, intracortical stimulation and mental health practices, task-oriented training, mirror therapy, among others^3–6. Virtual reality (VR) is a therapeutic approach that has been used in the field of rehabilitation in recent years. In this approach, users interact with virtual objects through the movements of their hands and body, or through tactile interfaces (gloves, joysticks, mouse), performing actions in a simulated environment⁷. The invention of low-cost human movement sensors in commercial game systems has made it easier to use video games for rehabilitation⁸. Examples of these systems include the Kinect for Microsoft's Xbox^®, the Nintendo Wii^® and the PlayStationMove^® by Sony.

The use of VR has increased the potential for motor learning and neuroplasticity during rehabilitation. A study using magnetic resonance imaging found consistent results, showing a reorganization of the sensorimotor cortex⁹. Functional improvements have also been associated with the use of VR by rehabilitation professionals. Allain et al.¹⁰ pointed out that performing a task in a virtual kitchen anticipated carrying out the task in real life. The virtual practice of shopping for groceries is associated with an improvement in the performance of actual grocery shopping¹¹, and patients who interact with the virtual world have fewer limitations when performing daily activities¹².

When VR has been applied in the rehabilitation of patients after strokes, it has mainly been used to help in the functional recovery of upper limbs, cognitive function, posture control and balance^13,14. Deficiencies in the upper limb after strokes may negatively impact the daily life of patients, by limiting their ability to carry out essential tasks that are necessary for an independent life¹⁵. The VR offers a rich environment in which patients may, after a stroke, solve problems and develop new abilities¹⁶. Considering the above, this study aimed to review and analyze information from the literature on the main intervention protocols delineated by rehabilitation professionals, using VR in commercial video game systems for the treatment of patients who had suffered strokes, while also identifying the most common outcomes found by these professionals.

METHODS

This was an integrative literature review, consisting of the analysis of relevant research that gave support for decision-making and the improvement of clinical practice, thus offering instruments for evidence-based practices¹⁷. An integrative review allows one to summarize the existing knowledge regarding a certain theme and identify how the data from previous publications can be used, in addition to pointing out the gaps in the knowledge that need to be researched by other studies¹⁷.

This review was elaborated in the following stages, as suggested by Souza et al.¹⁷: delimitation of the theme, hypothesis of the integrative review and objectives; determination of inclusion and exclusion criteria (sample selection); establishment of which data banks should be researched; definition of what information was to be extracted from selected articles; result analysis; discussion and presentation of the results; and, finally, the presentation of the review.

The research took place in April 2018, and the criteria for the inclusion of the studies were: articles that used commercial video game system VR in the rehabilitation of patients who had strokes; were published in 2011 or later; and had their entire content made available. There was no restriction regarding language; however, the Portuguese and English languages were chosen as the basis of the theoretical framework of this review. The bibliographical research was conducted in the LILACS and Medline (PubMed) databanks. The following descriptors, available on DeCS (the Health Science Descriptors website), were chosen for the search of the databanks: virtual reality exposure therapy; videogames; rehabilitation.

The research was conducted by two independent researchers, using the same descriptors, filters and databanks. Filters were used to exclude articles that did not have abstracts and those that were themselves literature reviews. After the filters were used, the abstract and content of the articles were carefully analyzed by two independent researchers who excluded studies with the following traits: interventions that were not focused on rehabilitation; development of intervention protocols in computer games; studies whose participants had primary health conditions different from those of stroke patients; studies whose games were classified as Exergames; articles with qualitative methodologies; and studies that used the video game in the residence of the participants and not in a clinical environment, which is known as telerehabilitation.

RESULTS

Characteristics of included studies

The research strategy using the different descriptors and combinations resulted in 1,396 studies on the theme, from which 1,355 were left after duplicates were excluded. From these, 883 were excluded as they did not make their complete content available, 441 were excluded after their title and abstract were read, 10 were removed after the entire text had been analyzed and eight because they had been published in other languages. There were 13 studies left, which were selected. The main reasons for the exclusion of the articles were: research focused on Exergames and telerehabilitation, research that applied VR to patients with conditions other than a stroke, and research that used computer games and not commercial gaming systems. The Figure is a flow chart detailing the selection of the studies.

Figure 1
Flowchart of studies found and selected after the application of inclusion and exclusion criteria.

Information regarding the title, authors, year of publication and periodical where the studies were published are shown in Table 1. Most of the selected articles had been published in 2014 (30.77%) and 38.46% were published in the years 2015, 2016 and 2017. The studies had been published in 11 different scientific journals (periodicals and magazines), from which the Neurorehabilitation and the Journal of Stroke and Cerebrovascular Diseases stood out, with two published articles each, meeting the inclusion criteria of this research.

Table 1
Summary of the selected studies including title, author/year of publication and name of journal.

N°	Author/Year	Title	Journal
1	Ribeiro, Ferraz, Pedreira, et al. (2015)	Virtual rehabilitation via Nintendo Wii® and conventional physical therapy effectively treat post-stroke hemiparetic patients	Topics in Stroke Rehabilitation. 2015; 22(4): 299-305
2	Sin, Lee (2013)	Additional virtual reality training using Xbox Kinect in stroke survivors with hemiplegia	American Journal of Physical Medicine and Rehabilitation. 2013; 92(10): 871-80
3	Fernandes, Passos, Brito, Campos (2014)	Comparison of the immediate effect of the training with a virtual reality game in stroke patients according side brain injury	NeuroRehabilitation. 2014; 35(1): 39–45
4	Cho, Lee, Song (2012)	Virtual-reality balance training with a video-game system improves dynamic balance in chronic stroke patients	Tohoku J Exp Med. 2012; 228(1): 69-74
5	Viana, Laurentino, Souza, et al. (2014)	Effects of the addition of transcranial direct current stimulation to virtual reality therapy after stroke: a pilot randomized controlled trial	NeuroRehabilitation. 2014; 34(3): 437–446.
6	Singh, Mohd Nordin, Aziz, et al. (2013)	Effects of substituting a portion of standard physiotherapy time with virtual reality games among community-dwelling stroke survivors	BMC Neurology. 2013; 13: 199.
7	Morone, Tramontano, Iosa, et al. (2014)	The efficacy of balance training with video game-based therapy in subacute stroke patients: a randomized controlled trial	Biomed Research International. 2014; Article ID 580861, 6 pages.
8	Rand, Givon, Weingarden, et al. (2014)	Eliciting upper extremity purposeful movements using video games: a comparison with traditional therapy for stroke rehabilitation	Neurorehabilitation and Neural Repair. 2014; 28(8): 733-739.
9	Hurkmans, Ribbers, Streur-Kranenburg, et al. (2011)	Energy expenditure in chronic stroke patients playing Wii Sports: a pilot study	Journal of Neuroengineering and Rehabilitation. 2011; 8: 38.
10	Carregosa, Santos, Masruha, et al. (2018)	Virtual rehabilitation through Nintendo Wii in poststroke patients: follow-up	Journal of Stroke and Cerebrovascular Diseases. 2018; 27(2): 494-498.
11	Lee, Shin, Song (2016)	Canoe game-based virtual reality training to improve trunk postural stability, balance, and upper limb motor function in subacute stroke patients: a randomized controlled pilot study	Journal of Physical Therapy Science. 2016; 28(7): 2019-24.
12	Fonseca, Silva, Pinto (2016)	Therapeutic effect of virtual reality on post-stroke patients: randomized clinical trial	Journal of Stroke and Cerebrovascular Diseases. 2016; 26(1): 94-100.
13	Lee, Huang, Ho, et al. (2017)	The effect of a virtual reality game intervention on balance for patients with stroke: a randomized controlled trial	Games for Health Journal. 2017; 6(5): 303-311.

Table 2 shows the data on the design, sample, video game system and games used in the articles selected for this review.

Table 2
Characterization of the selected articles regarding type of study, age group, sample, video game system and games.

N°	Methodological design	Age group	Sample	Sex	Time since stroke*	Video game	Games
1	Randomized Clinical Study	Adults	30	11 M; 19 F	Chronic	Nintendo Wii®	Tennis and Hula Hoop in the 1st weekly session and Football and Boxing in the 2nd.
2	Randomized Clinical Trial	Elders	35	20 M; 15 F	Chronic	Xbox 360®	Boxing, Bowling, Rally Ball, 20,000 Leaks and Space Pop.
3	Quasi-Experimental Clinical Trial	Adults up to 65 years of age	40	Not informed	Chronic	Xbox360®	Paddle Panic Mini
4	Randomized Clinical Trial	Adults and elders	22	14 M; 8 F	Chronic	Nintendo Wii®	Balance Bubble, Ski Slalom, Ski Jump, Football, Table Tiling and Penguin Slide
5	Randomized Clinical Trial	Adults above 21 years of age	20	16 M; 4 F	Chronic	Nintendo Wii®	Wii Sports Resort, Wii Play Motion and Let's Tap
6	Quasi-Experimental Clinical Trial	Adults above 55 years of age	28	16 M; 12 F	Chronic	Nintendo Wii® and Xbox 360®	Balance Bubble in Nintendo Wii Fit Plus, Balance Board and Rally Ball in Xbox 360
7	Randomized Clinical Study	Adults and elders	50	Not informed	Acute	Nintendo Wii®	Hula Hoop, Bubble Blower and Ski Slalom
8	Randomized Clinical Trial	Adults and elders	29	17 M; 12 F	Chronic	Xbox 360®, Ps2, Ps3 and SeeMe	Bowling and 20,000 Leaks in Xbox; Kung Foo and Slap Stream in Ps2; Start the Party in Ps3 and Ball or Cleaner in SeeMe
9	Quasi-Experimental Clinical Trial	Adults and elders	10	6 M; 4 F	Chronic	Nintendo Wii®	Tennis and Boxing
10	Quasi-Experimental Clinical Trial	Adults	5	3 M; 2 F	Chronic	Nintendo Wii®	Tennis and Hula Hoop in the 1st weekly session and Football and Boxing in the 2nd.
11	Randomized Clinical Trial	Elders	10	5 M; 5 F	Acute	Nintendo Wii®	Canoeing
12	Randomized Clinical Trial	Adults up to 65 years of age	27	11 M; 19 F	Chronic	Nintendo Wii®	Tennis and Hula Hoop in the 1st weekly session and Football and Boxing in the 2nd.
13	Randomized Clinical Trial	Adults and elders	47	34 M; 13 F	Chronic	Xbox360®	Darts, Golf, Bowling, Virtual Smash, Light Race, Space Pop, Rally Ball, Table Tennis and River Rush.

• M: male; F: female;
• ^*
  Strokes were considered chronic if more than six months had passed before the time of the study.

Most of the case studies selected (n = 9) were randomized clinical trials and four of them were quasi-experimental clinical trials. Studies #3 and #6 used a control group, with no participant randomization; studies #9 and #10 had no control group for comparison of the results. No study had gender restrictions regarding the composition of its sample. Most researchers analyzed a sample that included only adults (6) or included adult and elder (5) populations in their samples, and an age group varying from 18 to 85 years of age. Only two studies analyzed a sample comprising elders alone. The number of participants varied from 5-47. As for the time passed since the stroke, 11 studies were conducted with chronic patients, while two included patients who had a stroke less than six months prior.

The most common video game system was the Nintendo Wii^® (n = 8), followed by the Xbox 360^® (n = 3). Two studies combined different video game systems in their intervention protocols. One study used the Nintendo Wii^® and the Xbox 360^®, while the other used the Xbox 360^® and the Playstation^® SeeMe during intervention. The games chosen in the intervention protocol were mainly sports games, most frequently boxing, tennis, football and bowling.

Main outcomes, intervention protocols and results from the virtual reality activities in patients after a stroke

Table 3 is a summary of the main objectives, outcomes and assessment measures, intervention protocols and results described in the articles selected for this review. These studies had varying objectives and presented, especially, (a) objectives regarding the effects of VR on different physical and functional aspects of the patients or (b) compared the efficiency of VR to that of traditional treatments and/or different combined intervention techniques.

Table 3
Characterization of selected articles according to objectives, type of intervention and main results.

N°	Objectives	Outcomes/Evaluations	Intervention	Results Found
1	Comparing the effects of rehabilitation using the Nintendo Wii® to those of conventional therapy in order to improve the sensorimotor function and QoL of hemiplegic patients after a stroke.	Quality of Life: SF-36.	Two weekly sessions of 60 minutes each, for two months.	Both treatments improved the motor function in passive movements, pain, UL movement, balance and overall score.
		Motor function: Fugl-Meyer.	CG: CSPT	Both groups improved their QoL in the domains of functional capacity, vitality, physical and emotional aspects. The IG group improved their QoL in the mental health domain.
			IG: Nintendo Wii®	The IG group had a greater improvement in the functional capacity domain than the CG.
2	Investigate the effects of VR training using an Xbox® with the Kinect in the UL function, including movement amplitude, motor function and manual dexterity in hemiplegic patients after a stroke.	Motor function: Fugl-Meyer.	Three weekly sessions of 30 minutes each, for six weeks.	Both treatments improved motor function, dexterity and movement amplitude.
		Manual dexterity: Box and Blocks Test.	CG: COT	The IG showed a significant improvement in movement amplitude of UL for flexion, extension, shoulder abduction and elbow flexion.
		Movement amplitude: goniometry.	IG: Xbox® with a Kinect + COT	The results of motor function and dexterity were also significantly better in the IG.
3	Comparing the immediate effect of training with a VR game in stroke patients, according to the side of the brain lesion.	Movement amplitude: cinematic activity evaluation of drinking a glass of water.	Two 45 second series with a 15-minute resting time between them, one session.	After training, the two groups of patients and of healthy individuals improved their performance in the game.
		Performance in the game	CG1: right upper limb healthy	Patients with brain lesions on the right side (left hemiplegia) improved their shoulder flexion and elbow extension angles and did not show differences when compared with healthy individuals.
			CG2: left upper limb healthy	The group of patients with a brain lesion on the left side improved their movement amplitude but remained different from the healthy individuals.
			IG1: brain lesion on the left
			GI2: brain lesion on the right
4	Investigating the effects of the VR in the balance training using balance boards in chronic stroke patients.	Static balance: posturography	Three weekly sessions of 30 minutes each, for six weeks.	There was no static balance change in either group.
		Dynamic balance: Berg Balance Scale and Timed Up and Go Test	CG: SR	Both groups improved their dynamic balance.
			IG: Wii Fit + SR	The dynamic balance of the IG group improved more than that of the CG.
5	Determining whether transcranial direct-current stimulation (tDCS), combined with VR, is better than a rehabilitation using the Wii alone to improve the upper limb function and QoL in stroke patients	Motor function: Fugl-Meyer and Wolf Motor Function Test	VR: Three weekly sessions of 30 minutes each, for five weeks.	Both groups showed a significant improvement in all outcomes evaluated (motor function, spasticity, grip force and QoL), except in the upper limb domain of the QoL scale (SSQOL-UL).
		Spasticity: Modified Ashworth Scale	tDCS: Three weekly sessions of 13 minutes each, for five weeks.	There was a difference between the groups only in what concerns fist spasticity, in which more than 50% IG participants met a minimal clinically important difference.
		Grip force: Saehan manual dynamometer	CG: VR with Wii Fit + tDCS placebo
		QoL: SSQOL	IG: VR with Wii Fit + tDCS
6	Determining whether there were changes in the physical function and in daily living activities when part of the standard physical therapy treatment was replaced by virtual reality games for stroke survivors in the community.	Physical function: Timed Up and Go; 30 second sitting test; 10-minute walking test; six-minute walking test	Two weekly sessions of 120 minutes each, for six weeks.	Both groups improved their physical function (Timed Up and Go and 30-second sitting and getting up test).
		DLA: Barthel Scale	CG: 120 minutes of CSPT	Other outcomes did not improve (walking test - 10 and 6 minutes; DLA and static balance) in either group.
		Static balance: Probalance board	IG: 30 minutes of VR + 90 minutes of CSPT	There were no significant differences between groups.
7	Investigating the efficiency of balance training using the VR in the functional balance and in the disability of hemiplegic individuals, in the sub-acute stage of the stroke.	Dynamic Balance: Berg Balance Scale	Three weekly sessions of 40 minutes each, for four weeks.	Both groups showed improvements in all outcomes (dynamic balance, DLA, walking speed and ability to walk).
		DLA: Barthel Scale	CG: 20 minutes of CSPT	The IG had significant improvements in the dynamic balance, DLA and ability to walk, when compared to the CG.
		Walking speed: 10-meter walk test	IG: 20 minutes of VR + 20 minutes of CSPT	There was no difference in walking speed between the groups.
		Ability to walk: Functional Ambulation Category
8	Comparing the number of intentional and unintentional upper limb repetitions and evaluating the acceleration of the movement using the accelerometer in individuals with chronic stroke while playing a video game or undergoing traditional therapies.	Number of repetitions: video recording and observation	Two weekly sessions of 60 minutes each, for three months.	The number of intentional repetitions in the IG was five times higher than that of the CG.
		Acceleration and intensity of upper limb movement: accelerometer placed on the wrist	CG: COT	The acceleration of the movement in the IG was double the one in the CG.
			IG: VR	Patients with more intentional repetition had more acceleration and movement intensity.
9	Investigating the energy expenditure of patients with chronic stroke as they play table tennis and boxing in Wii Sports.	Energy expenditure: portable gas analyzer to measure oxygen uptake (VO2) (metabolic equivalent)	One session, 15 minutes for each game, with a 10-minute pause between games.	The participants of both games reached metabolic values that indicate high energy expenditure (equal or higher to three) when compared to basal assessments, with the exception of one patient in the group who played tennis.
			No CG.	There was no difference in the energy expenditure between the groups.
			Tennis group and boxing group.
10	Evaluating the sensorimotor and QoL recovery after a two-month rehabilitation using Nintendo Wii and determining whether what was learned through the technique is anchored.	Sensorimotor changes: Fugl-Meyer	Two weekly sessions of 60 minutes each, for two months.	The upper limb motor function score, the Fugl-Meyer total score and the SF-36 questionnaire functional capability domain did not change, indicating that the effects of the two-month therapy with the Nintendo Wii was sustained.
		QoL: SF-36	No CG.
			IG: 10 minutes stretch + 50 minutes VR
11	Investigating the effects of a VR program based on the canoeing game for torso posture control, balance and upper limb motor function after a stroke	Postural control: Trunk Impairment Scale, Functional Reach Test	Three weekly sessions of 30 minutes each, for four weeks.	The IG improved postural control and both groups improved functional reach. The IG had higher scores in postural control when compared to the CG.
		Balance: Berg Balance Scale and Timed Up and Go test	CG: CRP	Both groups improved balance, and the IG showed higher balance scores when compared with the CG.
		Motor Function: Fugl-Meyer.	IG: CRP + VR	The functional movement amplitude scores were significantly better in both groups, though the IG had better scores than the CG.
12	Investigating the therapeutic effects of VR associated with conventional physical therapy on balance while walking, and on the event of falls in stroke patients.	Balance while walking: Dynamic Gait Index	Two weekly sessions of 60 minutes each, for two weeks	Dynamic Gait Index scores improved in both groups, but only the CG showed a significant difference before and after the treatment.
		Falls: number of falls (occurrence).	CG: 60 minutes of CSPT	The number of falls after intervention diminished, but only in the IG was this difference statistically significant.
			IG: 10 minutes of CSPT + 45 minutes of VR	There were no significant differences between groups.
13	Investigating the effects of VR on balance training using the Kinect for Xbox games in chronic stroke patients.	Balance: Berg Balance Scale, Functional Reach Test and Timed Up and Go – cognition	Two weekly sessions of 90 minutes each, for six weeks	Both groups significantly improved their balance, the Balance Scale scores and the Timed Up and Go – cognition. The Functional Reach Test scores did not improve.

• QoL: quality of life; CSPT: Conventional or standard physical therapy; CG: Control group; IG: Intervention or experimental group; COT: Conventional or standard Occupational Therapy; SR: Standard Rehabilitation, defined by the author as 30 minutes of physical therapy, 30 minutes of occupational therapy and speech-language therapy if needed4; DLA: Daily Living Activities; CRP: Conventional Rehabilitation Program, defined by the author as 30 minutes of physical therapy, 30 minutes of occupational therapy and 15 minutes of Functional Electrical Stimulation10; SRP: Standardized Rehabilitation Program, characterized by the authors as focused on increasing strength, resistance training, walking training and DLA training13. SSQOL: Stroke-Specific Quality of Life Scale

Among the analyzed investigations, the main outcomes evaluated were balance (six studies), upper limb motor function (five studies), quality of life (four studies) and daily living activities (DLA) in three studies. Other, less frequent outcomes evaluated included dexterity evaluations, movement amplitude, spasticity, grip force, physical function, falls, walking, acceleration and movement repetition, energy expenditure, postural control and performance in the video game.

Regarding the assessment of the outcomes, there was a great variety of tests and instruments used to evaluate the participants in the studies. The most commonly used were the Fugl-Meyer Assessment (five studies); the Berg Balance Scale and the Timed Up and Go test, both used in four studies; the Barthel Index (three studies); and the SF-36 Questionnaire, the 10-minute Walk Test and the Functional Reach Test, each used in two studies.

Regarding the intervention protocol, two studies evaluated the effects of an intensive single-session training; six and five studies, respectively, applied VR two or three times a week. The length of each session varied from 30 to 120 minutes, lasting from 30 to 60 minutes in most protocols (four studies each). The intervention also lasted from two to 12 weeks, although six was the total duration of the intervention in most studies (four studies), followed by four and eight weeks (two studies each).

Regarding studies that investigated the effects of the VR on different physical and functional aspects of the patients, most indicated the efficiency of VR regarding the outcomes of dynamic balance (five studies), upper limb motor function (four studies) and quality of life (three studies). Improvements in other outcomes such as dexterity, spasticity, grip force, physical function, walking, DLA, repetition and acceleration of movement, energy expenditure, posture control and falls, were each described in only one study.

The results comparing the VR with conventional treatments indicated that VR is more efficient, especially in the dynamic balance outcome (three studies). For most studies, conventional or standard treatments were those that described weekly sessions of physical or occupational therapy, lasting 30 minutes. For the group of patients who were submitted to VR therapy, each of the following outcomes were significant in only one study: quality of life, movement amplitude, upper limb motor function, dexterity, spasticity, DLA, walking, repetition and acceleration of movement and posture control.

Some studies indicated no difference between the benefits of the control group (under conventional treatment) and the intervention group (VR). The outcomes that had the same results comparing the groups were static balance, DLA, walking and dynamic balance, each mentioned in two studies, and quality of life, mentioned in one.

DISCUSSION

Virtual reality is a technique used in multiple areas and comprises various equipment, hardware and software. It has been applied in research, education, evaluation and therapeutic intervention or rehabilitation¹⁸. The growing number of publications in recent years, especially since 2014, indicates the interest of researchers and clinicians in VR. This increase may be related to the improvement and popularization of this technology, which has enabled professionals to access digital media, directly influencing the studies that use this technique¹⁹.

This review sought to analyze intervention protocols used by rehabilitation professionals, who used VR in commercial video game systems. In this context, the studies analyzed were mostly randomized clinical trials, which were created specifically to analyze the effects of a specific intervention²⁰. Thus, randomized clinical trials were, among primary studies, the most relevant for the objectives of this review.

The predominance of studies, with a sample comprised specifically of adults, is in line with the epidemiological changes in the incidence of stroke in the last decades. Although the occurrence of stroke is associated with aging, data from 119 countries has shown that the number of cases in the age group 20-64 increased from 25% in 1990 to 31% in 2010²¹.

Most analyzed studies evaluated the efficiency of VR in patients with chronic strokes, though the rehabilitation techniques commonly used in this population are focused on the first three months, that is, in the acute stage of the disease^22,23. The results of this review reiterate the recent change in the rehabilitation intervention paradigm, regarding the time required for a possible motor recovery. Recent studies have presented new experiences that have demonstrated functional gains, even after the spontaneous recovery period^24,25.

The results of this review showed that the Nintendo Wii^® was the most commonly used video game system in the articles analyzed. The Nintendo Wii^® is a low-cost piece of equipment that is easy for the therapist to handle and can be operated independently by the patient²⁶. Another commercial video game system used in the protocol of the studies was the Xbox 360^®, used in the interventions of five studies. It was used in isolation in three of these studies and combined with other systems in the other two. The fact that the Xbox 360^® was used less frequently in the intervention protocols may partially be explained by the fact that it has been on the market for less time²⁷. The Xbox 360^® was released in 2010, while the Nintendo Wii^® has been on the market since 2006. Nonetheless, the Xbox 360^® has advantages when compared with the Nintendo Wii^®, as it has a more advanced technology, more realistic and less child-targeted characters, in addition to allowing the interaction of the user with the virtual environment without the need of a manual controller²⁷. As a consequence, the Xbox 360^® may offer a better game interface with the player and has broader applications in the therapeutic process, for instance, for patients whose manual functions are more compromised, as no object needs to be gripped throughout the game.

Sports games were the most frequently used in the analyzed studies, which may be related to the main outcomes evaluated in the articles. Balance and motor function were the main outcomes evaluated in the studies that were part of this review, followed by quality of life and DLA. The other evaluated variables were related to dexterity, movement amplitude, spasticity, grip force, physical function, walking and falls. These findings indicate that most studies investigated the effects of VR on outcomes related to structural components and body functions, especially regarding the vestibular system and neuromusculoskeletal functions.

According to the World Health Organization (WHO)²⁸, a health condition such as a stroke can impact different areas and health components of the health and life of a person. Body functions are physiological functions of the body systems and structures involving the anatomic parts of organs and limbs²⁸. These are components of human functionality that may be impacted by a stroke. In this review, few studies analyzed the effect of VR on the components of activity and participation. The WHO²⁸ describes activity as the execution of a task or action by an individual, while participation is this individual's involvement in daily living situations. Considering these concepts, the activity component was investigated in only three studies, and four studies evaluated the quality of life outcome. However, despite the guidelines for adult stroke rehabilitation and recovery²⁵ recommending the evaluation of participation through measures of quality of life, the correlation of this information with participation should be done with care. Quality of life relates to how people “feel” about their health conditions or their consequences²⁸. Therefore, the quality of life is a construct of “subjective wellbeing” and functionality, on one hand but, on the other hand, refers to objective external signs from the individual²⁸. As guidance for the development of future studies, the WHO suggests the formulation of investigations that establish associations between the functionality components and the concepts of quality of life and subjective wellbeing²⁸.

The Fugl-Meyer Assessment, the Berg Balance Scale and the Timed Up and Go test were the most common tests in the studies, followed by the Barthel Scale, the SF-36, the 10-minute Walk Test and the Functional Reach Test. The Fugl-Meyer Assessment is largely used in literature to evaluate sensorimotor function in hemiplegic patients^29,30, and it shows high confidence levels, both intra- and inter-examiner, allowing its use as a clinical evaluation instrument and as a research instrument in stroke^29–31.

Balance evaluations were conducted, in most cases, using the Berg Balance Scale and the Timed Up and Go test, followed by the Functional Reach Test. The Berg Balance Scale was shown to have an excellent confidence level when it comes to stroke patients³². The Timed Up and Go and the Functional Reach Tests are simple and objective measures, easy to apply in clinical practice³³. The validity of these tests are good, with excellent intra- and inter-examiner confidence levels when applied to people after a stroke^29,34. The evaluation of the outcome “balance” was conducted combining at least two tests: three studies combined the Berg Balance Scale and the Timed Up and Go test, while two combined these two measurements with the Functional Reach Test. Recent studies highlighted that evaluations of static and dynamic balance are very often in agreement, reiterating the need for the use of the two tools simultaneously – especially the Berg Balance Scale and the Timed Up and Go test^33,35.

The Barthel Index was the most widely-used instrument to evaluate DLA and was identified in the literature as the second most used tool to evaluate the results of clinical trials after stroke. This index has favorable construct validity to measure activity according to the WHO, and its inter- and intra-examiner reliability is considered moderate to high³⁶.

Information on quality of life is an important outcome in the evaluation of stroke patients and in the use of different treatments. The SF-36 was used to evaluate quality of life in two studies in this review, while another two used specific instruments for stroke patients. The SF-36 is described in the literature as the most used generic instrument to measure quality of life in the general population³⁷, presenting a satisfactory internal consistency, as well as validity and test-retest reliability for the eight sub-assessments for stroke patients^37,38.

The analysis of the intervention protocols used in the studies included in this review showed great variability regarding the number of sessions, the length of each session and the total length of the treatment. However, the predominance of short-to-moderate length and low-intensity protocols stand out in regards to weekly sessions. Only two studies applied short high-intensity protocols (one session only). The intervention methodology of most studies was applying two sessions a week, lasting from 30 to 60 minutes each, throughout the six weeks.

Recent systematic reviews have also pointed out these differences in the intensity and length of the interventions that evaluated the efficiency of the VR^39,40. Laver et al.⁴¹ suggested that longer interventions (more than 15 hours overall) were the best option, as well as personalized virtual reality interventions; however, their findings were not statistically significant. Considering the diversity of intervention protocols, future studies need to analyze the prognosis of stroke patients who are going through rehabilitations using VR, to define the minimum length the intervention must have for improvements to be detected, and the ideal length of each session, thus making this technique more viable.

This review found important differences in the results, when the main outcomes of each study were compared. Dynamic balance, upper limb motor function, quality of life and DLA improved significantly in different studies. However, other studies did not find any improvement in the outcome of these variables after VR was used. The comparison between the efficiency of VR coupled with conventional treatments also showed an inconsistency between the evidences, as some investigations found that VR brought benefits, while others did not find differences between the results of the different groups. This result instability is also found in the literature. Li et al.⁴², through a meta-analysis, gave support to the use of VR to improve balance after strokes. However, Santos et al.⁴³ suggested that the data was not clear regarding the efficiency of VR to improve balance and functional independence. Lee¹³ indicated VR as an effective and clinically-applicable therapeutic strategy for the functional recovery of upper limbs in the treatment of stroke patients. On the contrary, Laver et al.⁴¹ found that the use of VR and interactive video games were not more beneficial than conventional therapeutic approaches to improve the function of upper limbs, balance or quality of life. These authors point out that VR may be beneficial to improve DLA only when used as an addition to conventional treatments, increasing the overall duration of the therapy⁴¹.

These results show that there are evidences suggesting the efficiency of VR in the rehabilitation of stroke patients, but that these are not sufficient to support, without doubt, their use in these cases. Therefore, the results found in the articles analyzed in this review must be interpreted carefully, especially regarding the differences in the samples, intensity and length of the therapies, which makes the studies more difficult to compare and less viable for generalization.

FINAL CONSIDERATIONS

This review found a large number of studies, published in 2004 or later, that had analyzed the use of VR in stroke patients, and used research designs that were appropriate to investigate evidence of efficiency. Adult chronic patients were the most prevalent in the samples, which varied greatly in size from one study to the other. The Nintendo Wii^® and sports games were the video game system and type of game most used during therapies. The main outcomes measured were balance, upper limb motor function, quality of life and DLA. Most protocols had short to moderate durations (six weeks) and low intensity regarding the number of weekly sessions (two sessions). The results varied with respect to the efficiency of VR, considering both the improvement in the outcomes and the comparison with traditional treatments.

This review contributes to a better understanding of VR and the use of this technique in a clinical context by rehabilitation professionals. The characteristics of the intervention protocols described in this study should be incorporated in the elaboration of new research on the prognosis of stroke patients who are using VR in their rehabilitation. It is also recommended that new research focuses on the efficiency of the VR regarding DLA and social participation.

References

• ¹
  Monteiro Junior RS, Carvalho RJ, Silva EB, Bastos FG. Efeito da reabilitação virtual em diferentes tipos de tratamento. Rev Bras Cienc Saúde. 2011;9(29):56-63. https://doi.org/10.13037/rbcs.vol9n29.1331
  » https://doi.org/10.13037/rbcs.vol9n29.1331
• ²
  Sampaio RF, Mancini MC, Fonseca ST. Produção científica e atuação profissional: aspectos que limitam essa integração na Fisioterapia e na Terapia Ocupacional. Braz J Phys Ther. 2002;6(3):113-8.
• ³
  Assis GA, Corrêa AG, Martins MB, Pedrozo WG, Lopes RD. An augmented reality system for upper-limb post-stroke motor rehabilitation: a feasibility study. Disabil Rehabil Assist Technol. 2016 Aug;11(6):521-8. https://doi.org/10.3109/17483107.2014.979330
  » https://doi.org/10.3109/17483107.2014.979330
• ⁴
  Bondoc S, Booth J, Budde G, Caruso K, DeSousa M, Earl B, et al. Mirror therapy and task-oriented training for people with a paretic upper extremity. Am J Occup Ther. 2018 Mar/Apr;72(2):p1. https://doi.org/10.5014/ajot.2018.025064
  » https://doi.org/10.5014/ajot.2018.025064
• ⁵
  Saunders DH, Sanderson M, Hayes S, Kilrane M, Greig CA, Brazzelli M, et al. Physical fitness training for stroke patients. Cochrane Database Syst Rev. 2016 Mar;3:CD003316. https://doi.org/10.1002/14651858.CD003316.pub6
  » https://doi.org/10.1002/14651858.CD003316.pub6
• ⁶
  Vasileva D, Lubenova D, Mihova M, Dimitrova A, Grigorova-Petrova K. Influence of Kinesitherapy on Gait in Patients with Ischemic Stroke in the Chronic Period. Open Access Maced J Med Sci. 2015 Dec;3(4):619-23. https://doi.org/10.3889/oamjms.2015.107
  » https://doi.org/10.3889/oamjms.2015.107
• ⁷
  Levin MF, Snir O, Liebermann DG, Weingarden H, Weiss PL. Virtual reality versus conventional treatment of reaching ability in chronic stroke: clinical feasibility study. Neurol Ther. 2012 Aug;1(1):3. https://doi.org/10.1007/s40120-012-0003-9
  » https://doi.org/10.1007/s40120-012-0003-9
• ⁸
  Kwon JS, Park MJ, Yoon IJ, Park SH. Effects of virtual reality on upper extremity function and activities of daily living performance in acute stroke: a double-blind randomized clinical trial. NeuroRehabilitation. 2012;31(4):379-85. https://doi.org/10.3233/NRE-2012-00807
  » https://doi.org/10.3233/NRE-2012-00807
• ⁹
  Schuster-Amft C, Henneke A, Hartog-Keisker B, Holper L, Siekierka E, Chevrier E, et al. Intensive virtual reality-based training for upper limb motor function in chronic stroke: a feasibility study using a single case experimental design and fMRI. Disabil Rehabil Assist Technol. 2015;10(5):385-92. https://doi.org/10.3109/17483107.2014.908963
  » https://doi.org/10.3109/17483107.2014.908963
• ¹⁰
  Allain P, Foloppe DA, Besnard J, Yamaguchi T, Etcharry-Bouyx F, Le Gall D, et al. Detecting everyday action deficits in Alzheimer's disease using a nonimmersive virtual reality kitchen. J Int Neuropsychol Soc. 2014 May;20(5):468-77. https://doi.org/10.1017/S1355617714000344
  » https://doi.org/10.1017/S1355617714000344
• ¹¹
  Kizony R, Mallory K, Sinoff G, Klinger E, Josman N. Using a virtual supermarket as a tool for training executive functions in people with mild cognitive impairment. J Stroke Cerebrovasc Dis. 2012;23(5):10-2.
• ¹²
  da Costa RM, de Carvalho LA. The acceptance of virtual reality devices for cognitive rehabilitation: a report of positive results with schizophrenia. Comput Methods Programs Biomed. 2004 Mar;73(3):173-82. https://doi.org/10.1016/S0169-2607(03)00066-X
  » https://doi.org/10.1016/S0169-2607(03)00066-X
• ¹³
  Lee KH. Effects of a virtual reality-based exercise program on functional recovery in stroke patients: part 1. J Phys Ther Sci. 2015 Jun;27(6):1637-40. https://doi.org/10.1589/jpts.27.1637
  » https://doi.org/10.1589/jpts.27.1637
• ¹⁴
  Saposnik G, Levin M, Thomas S, Deutsch JE, Crotty M; Outcome Research Canada (SORCan) Working Group. Virtual reality in stroke rehabilitation: a meta-analysis and implications for clinicians. Stroke. 2011 May;42(5):1380-6. https://doi.org/10.1161/STROKEAHA.110.605451
  » https://doi.org/10.1161/STROKEAHA.110.605451
• ¹⁵
  Basílio ML, de Faria-Fortini I, Polese JC, Scianni AA, Faria CD, Teixeira-Salmela LF. Handgrip strength deficits best explain limitations in performing bimanual activities after stroke. J Phys Ther Sci. 2016 Apr;28(4):1161-5. https://doi.org/10.1589/jpts.28.1161
  » https://doi.org/10.1589/jpts.28.1161
• ¹⁶
  Laver K, George S, Thomas S, Deutsch JE, Crotty M. Cochrane review: virtual reality for stroke rehabilitation. Eur J Phys Rehabil Med. 2012 Sep;48(3):523-30.
• ¹⁷
  Souza MT, Silva MD, Carvalho R. Revisão integrativa: o que é e como fazer. Einstein (Sao Paulo). 2010;8(1):102-6. https://doi.org/10.1590/s1679-45082010rw1134
  » https://doi.org/10.1590/s1679-45082010rw1134
• ¹⁸
  Dores AR, Barbosa F, Marques A, Carvalho IP, De Sousa L, Castro-Caldas A. [Virtual reality and rehabilitation: why or why not? A systematic literature review]. Acta Med Port. 2012 Nov-Dec;25(6):414-21. Portuguese.
• ¹⁹
  Caiana TL, Nogueira DL, Lima AC. A realidade virtual e seu uso como recurso terapêutico ocupacional: revisão integrativa. Cad Ter Ocup UFSCar. 2016;24(3):575-89. https://doi.org/10.4322/0104-4931.ctoAR0619
  » https://doi.org/10.4322/0104-4931.ctoAR0619
• ²⁰
  Portney LG, Watkins M. Foundations of clinical research: applications to practice. 3rd ed. Upper Saddle River: Prentice Hall; 2009.
• ²¹
  Feigin VL, Forouzanfar MH, Krishnamurthi R, Mensah GA, Connor M, Bennett DA, et al. Global and regional burden of stroke during 1990-2010: findings from the Global Burden of Disease Study 2010. Lancet. 2014 Jan;383(9913):245-54. https://doi.org/10.1016/S0140-6736(13)61953-4
  » https://doi.org/10.1016/S0140-6736(13)61953-4
• ²²
  Miller EL, Murray L, Richards L, Zorowitz RD, Bakas T, Clark P, et al. Comprehensive overview of nursing and interdisciplinary rehabilitation care of the stroke patient: a scientific statement from the American Heart Association. Stroke. 2010 Oct;41(10):2402-48. https://doi.org/10.1161/STR.0b013e3181e7512b
  » https://doi.org/10.1161/STR.0b013e3181e7512b
• ²³
  Gamba RT, Cruz DM. Efeitos da terapia por contensão induzida em longo prazo em pacientes pós AVC. Rev Neurocienc. 2011;19(4):735-40.
• ²⁴
  Belliveau D, Belliveau I, Camire-Raymond A, Kessler D, Egan M. Use of occupational performance coaching for stroke survivors (OPC-Stroke) in late rehabilitation: a descriptive case study. Open J Occup Ther. 2016;4(2):7. https://doi.org/10.15453/2168-6408.1219
  » https://doi.org/10.15453/2168-6408.1219
• ²⁵
  Winstein CJ, Stein J, Arena R, Bates B, Cherney LR, Cramer SC, et al. Guidelines for adult stroke rehabilitation and recovery: a guideline for healthcare professionals from the American Heart Association/American Stroke Association. Stroke. 2016 Jun;47(6):e98-169. https://doi.org/10.1161/STR.0000000000000098
  » https://doi.org/10.1161/STR.0000000000000098
• ²⁶
  Tripette J, Murakami H, Ryan KR, Ohta Y, Miyachi M. The contribution of Nintendo Wii Fit series in the field of health: a systematic review and meta-analysis. PeerJ. 2017 Sep;5:e3600. https://doi.org/10.7717/peerj.3600
  » https://doi.org/10.7717/peerj.3600
• ²⁷
  Mendes FA, Arduini L, Botelho A, Cruz MB, Paz CCS, Pompeu SMAA et al. Pacientes com a Doença de Parkinson são capazes de melhorar seu desempenho em tarefas virtuais do Xbox Kinect^®: “uma série de casos. Motricidade. 2015;11(3):68-80. https://doi.org/10.6063/motricidade.3805
  » https://doi.org/10.6063/motricidade.3805
• ²⁸
  Organização Mundial de Saúde – OMS. CIF: Classificação internacional de funcionalidade, incapacidade e saúde. São Paulo: Editora da Universidade de São Paulo; 2003.
• ²⁹
  Alt Murphy M, Resteghini C, Feys P, Lamers I. An overview of systematic reviews on upper extremity outcome measures after stroke. BMC Neurol. 2015 Mar;15(1):29. https://doi.org/10.1186/s12883-015-0292-6
  » https://doi.org/10.1186/s12883-015-0292-6
• ³⁰
  Santisteban L, Térémetz M, Bleton JP, Baron JC, Maier MA, Lindberg PG. Upper limb outcome measures used in stroke rehabilitation studies: a systematic literature review. PLoS One. 2016 May;11(5):e0154792. https://doi.org/10.1371/journal.pone.0154792
  » https://doi.org/10.1371/journal.pone.0154792
• ³¹
  Maki T, Quagliato EM, Cacho EW, Paz LP, Nascimento NH, Inoue MM, et al. Estudo de confiabilidade da aplicação da escala de Fugl-Meyer no Brasil. Braz J Phys Ther. 2006;10(2):177-83. https://doi.org/10.1590/S1413-35552006000200007
  » https://doi.org/10.1590/S1413-35552006000200007
• ³²
  Tyson SF, Connell LA. How to measure balance in clinical practice: a systematic review of the psychometrics and clinical utility of measures of balance activity for neurological conditions. Clin Rehabil. 2009 Sep;23(9):824-40. https://doi.org/10.1177/0269215509335018
  » https://doi.org/10.1177/0269215509335018
• ³³
  Woellner SS, Araujo AG, Cabral FM, Uessler PN, Soares AV. Testes de equilíbrio em pacientes hemiparéticos por AVC. Neurociencias. 2015;11(1):32-40.
• ³⁴
  Hafsteinsdóttir TB, Rensink M, Schuurmans M. Clinimetric properties of the Timed Up and Go Test for patients with stroke: a systematic review. Top Stroke Rehabil. 2014 May-Jun;21(3):197-210. https://doi.org/10.1310/tsr2103-197
  » https://doi.org/10.1310/tsr2103-197
• ³⁵
  Correia A, Pimenta C, Alves M, Virella D. Concordância entre instrumentos de avaliação do equilíbrio após acidente vascular cerebral. Saúde Tecnol. 2017;(17):33-8.
• ³⁶
  Taylor-Rowan M, Wilson A, Dawson J, Quinn TJ. Functional assessment for acute stroke trials: properties, analysis, and application. Front Neurol. 2018 Mar;9:191. https://doi.org/10.3389/fneur.2018.00191
  » https://doi.org/10.3389/fneur.2018.00191
• ³⁷
  de Haan RJ. Measuring quality of life after stroke using the SF-36. Stroke. 2002 May;33(5):1176-7. https://doi.org/10.1161/01.STR.0000015223.98452.97
  » https://doi.org/10.1161/01.STR.0000015223.98452.97
• ³⁸
  Anderson C, Laubscher S, Burns R. Validation of the Short Form 36 (SF-36) health survey questionnaire among stroke patients. Stroke. 1996 Oct;27(10):1812-6. https://doi.org/10.1161/01.STR.27.10.1812
  » https://doi.org/10.1161/01.STR.27.10.1812
• ³⁹
  Iruthayarajah J, McIntyre A, Cotoi A, Macaluso S, Teasell R. The use of virtual reality for balance among individuals with chronic stroke: a systematic review and meta-analysis. Top Stroke Rehabil. 2017 Jan;24(1):68-79. https://doi.org/10.1080/10749357.2016.1192361
  » https://doi.org/10.1080/10749357.2016.1192361
• ⁴⁰
  Yates M, Kelemen A, Sik Lanyi C. Virtual reality gaming in the rehabilitation of the upper extremities post-stroke. Brain Inj. 2016;30(7):855-63. https://doi.org/10.3109/02699052.2016.1144146
  » https://doi.org/10.3109/02699052.2016.1144146
• ⁴¹
  Laver KE, Lange B, George S, Deutsch JE, Saposnik G, Crotty M. Virtual reality for stroke rehabilitation. Cochrane Database Syst Rev. 2017 Sep;(9):CD008349. https://doi.org/10.1002/14651858.CD008349.pub2
  » https://doi.org/10.1002/14651858.CD008349.pub2
• ⁴²
  Li Z, Han XG, Sheng J, Ma SJ. Virtual reality for improving balance in patients after stroke: a systematic review and meta-analysis. Clin Rehabil. 2016 May;30(5):432-40. https://doi.org/10.1177/0269215515593611
  » https://doi.org/10.1177/0269215515593611
• ⁴³
  Santos LRA, Carregosa AA, Masruha MR, Santos PA, Coêlho MLS, Ferraz DD, et al. The use of Nintendo Wii in the rehabilitation of poststroke patients: a systematic review. J Stroke Cerebrovasc Dis. 2015 Oct;24(10):2298-305. https://doi.org/10.1016/j.jstrokecerebrovasdis.2015.06.010
  » https://doi.org/10.1016/j.jstrokecerebrovasdis.2015.06.010

Publication Dates

• Publication in this collection
  13 May 2019
• Date of issue
  Apr 2019

History

• Received
  15 Sept 2018
• Reviewed
  09 Nov 2018
• Accepted
  10 Dec 2018