ABSTRACT.
Verbal fluency tests are widely used in neuropsychological assessment. The quantitative and qualitative analysis of the performance of aphasic individuals on the phonemic verbal fluency (PVF) test can contribute to a better understanding of cognitive changes in this group of patients.
Objective:
This study aimed to analyze clustering and switching measures of PVF in people with aphasia and investigate the relationship between the use of these strategies, the quantitative performance on the test, and the performance on executive functions.
Methods:
This is a cross-sectional study of 15 people with aphasia, right-handed, native speakers of Brazilian Portuguese, aged 19 to 92 years. They were submitted to the F-A-S test and the Clock Drawing Test (CDT). The following measures were obtained in the F-A-S test: total score, number of clusters, mean cluster size, and number of switches. Spearman's test was used to analyze correlation.
Results:
We found a positive correlation among all F-A-S test scores, ranging from p=0.61 (p<0.001) to p=0.91 (p<0.001). No correlation was identified between these measures and CDT performance (p≤0.31; p≥0.260).
Conclusions:
The quantitative and qualitative analysis of F-A-S in people with aphasia, even those with different linguistic manifestations, showed that these individuals presented lower scores and that the number of total words and the number of switches were strongly correlated. We found no correlation between executive function, assessed by the CDT, and switching performance on the F-A-S test.
Keywords:
language tests; neuropsychological tests; cognition; language; aphasia
RESUMO.
Testes de fluência verbal são procedimentos amplamente utilizados na avaliação neuropsicológica. A análise da performance de pessoas com afasia no teste de fluência verbal fonêmica (FVF), do ponto de vista quantitativo e qualitativo, pode contribuir para uma melhor compreensão das mudanças cognitivas que ocorrem nesse grupo de pacientes.
Objetivo:
Os objetivos deste estudo foram analisar as medidas de estratégias de agrupamento e de mudança de agrupamento na FVF de pessoas com afasia e investigar a correlação entre o uso dessas estratégias, o desempenho quantitativo no teste e o desempenho em funções executivas. Métodos: Foi realizado um estudo transversal com 15 pessoas com afasia, destros, falantes nativos do português brasileiro, com idade entre 19 e 92 anos. Eles foram submetidos ao teste F-A-S e ao Teste do Relógio (TDR). Foram obtidas as seguintes medidas no F-A-S: escore total, número e tamanho médio de agrupamento e número de mudança de agrupamento. Para análise de correlação foi utilizado o teste de Spearman.
Resultados:
Foi verificada correlação positiva entre todos os escores do teste F-A-S, variando entre p=0,61 (p<0,001) e p=0,91 (p<0,001). Não foram encontradas correlações entre tais medidas e o escore no TDR (p≤0,31; p≥0,260).
Conclusões:
A análise quantitativa e qualitativa do F-A-S de pessoas com afasia, mesmo com diferentes manifestações linguísticas, mostrou que elas apresentam escores inferiores e o número total de palavras e o número de mudanças de agrupamento possuem forte correlação. Não foi observada correlação entre funções executivas, avaliada pelo TDR, e o desempenho na medida de mudança de agrupamento.
Palavras-chave:
testes de linguagem; testes neuropsicológicos; cognição; idioma; afasia
INTRODUCTION
Verbal fluency tests (VFTs) are widely used in neuropsychological assessments to identify cognitive disorders due to neurological injuries, such as dementia, traumatic brain injury, stroke, and tumor, and other neurological or psychiatric conditions.11. Ruff RM, Light RH, Parker SB, Levin HS. The psychological construct of word fluency. Brain Lang. 1997;57(3):394-405. https://doi.org/10.1006/brln.1997.1755
https://doi.org/10.1006/brln.1997.1755...
,22. Machado TH, Fichman HC, Santos EL, Carvalho VA, Fialho PP, Koenig AM, et al. Normative data for healthy elderly on the phonemic verbal fluency task-F-A-S. Dement Neuropsychol. 2009;3(1):55-60. https://doi.org/10.1590/S1980-57642009DN30100011
https://doi.org/10.1590/S1980-57642009DN...
Although VFTs are very useful in identifying lexical difficulties, other important cognitive aspects can be analyzed during its application. Firstly, some authors conceptualized VFTs as instruments to measure executive function.33. Lezak MD, Howieson DB, Biegler ED, Tranel D. Neuropsychological Assessment. 2. ed. New York: Oxford University Press; 2012. Then, other researchers assumed that VFTs were only loaded onto the language factor.44. Whiteside DM, Kealey T, Semla M, Luu H, Rice L, Basso MR, et al. Verbal fluency: language or executive function measure? Appl Neuropsychol Adult. 2016;23(1):29-34. https://doi.org/10.1080/23279095.2015.1004574
https://doi.org/10.1080/23279095.2015.10...
Although VFTs are strongly associated with lexical ability and access, lexical-semantic network, word and semantic knowledge, and auditory attention, they are also related to multiple aspects of executive functions.55. Aita SL, Beach JD, Taylo SE, Borgogna NC, Harrell MN, Hill BD. Executive, language, or both? An examination of the construct validity of verbal fluency measures. Appl Neuropsychol Adult. 2019;26(5):441-51. https://doi.org/10.1080/23279095.2018.1439830
https://doi.org/10.1080/23279095.2018.14...
The three forms of VFTs assessed are phonemic, verbal, and semantic. In these tests, participants are asked to provide responses in a limited time frame (usually one minute) according to the form being used.22. Machado TH, Fichman HC, Santos EL, Carvalho VA, Fialho PP, Koenig AM, et al. Normative data for healthy elderly on the phonemic verbal fluency task-F-A-S. Dement Neuropsychol. 2009;3(1):55-60. https://doi.org/10.1590/S1980-57642009DN30100011
https://doi.org/10.1590/S1980-57642009DN...
,66. Senhorini MC, Amaro Júnior E, Ayres AM, Simone A, Busatto GF. Phonemic fluency in Portuguese-speaking subjects in Brazil: ranking of letters. J Clin Exp Neuropsychol. 2006;28(7):1191-200. https://doi.org/10.1080/13803390500350969
https://doi.org/10.1080/1380339050035096...
,77. Casals-Coll M, Sánchez-Benavides G, Quintana M, Manero RM, Rognoni T, Calvo L, et al. Estudios normativos españoles em población adulta joven (proyecto NEURONORMA jóvenes): normas para lós test de fluencia verbal. Neurología. 2013;28(1):33-40. https://doi.org/10.1016/j.nrl.2012.02.010
https://doi.org/10.1016/j.nrl.2012.02.01...
In the semantic fluency test, the individuals have to produce as many words as possible belonging to a certain semantic category predefined by the examiner (e.g., animals). In the fluency task with verbal criteria, only verbs can be produced. Finally, in orthographic or phonemic verbal fluency (PVF), the participants have to produce words beginning with a specific letter or phoneme.22. Machado TH, Fichman HC, Santos EL, Carvalho VA, Fialho PP, Koenig AM, et al. Normative data for healthy elderly on the phonemic verbal fluency task-F-A-S. Dement Neuropsychol. 2009;3(1):55-60. https://doi.org/10.1590/S1980-57642009DN30100011
https://doi.org/10.1590/S1980-57642009DN...
,66. Senhorini MC, Amaro Júnior E, Ayres AM, Simone A, Busatto GF. Phonemic fluency in Portuguese-speaking subjects in Brazil: ranking of letters. J Clin Exp Neuropsychol. 2006;28(7):1191-200. https://doi.org/10.1080/13803390500350969
https://doi.org/10.1080/1380339050035096...
,88. Steiner VAG, Mansur LL, Brucki SMD, Nitrini R. Phonemic verbal fluency and age: a preliminary study. Dement Neuropsychol. 2008;2(4):328-32. https://doi.org/10.1590/S1980-57642009DN20400017
https://doi.org/10.1590/S1980-57642009DN...
The PVF F-A-S test is one of the most used instruments to assess people with neurological disorders.22. Machado TH, Fichman HC, Santos EL, Carvalho VA, Fialho PP, Koenig AM, et al. Normative data for healthy elderly on the phonemic verbal fluency task-F-A-S. Dement Neuropsychol. 2009;3(1):55-60. https://doi.org/10.1590/S1980-57642009DN30100011
https://doi.org/10.1590/S1980-57642009DN...
,66. Senhorini MC, Amaro Júnior E, Ayres AM, Simone A, Busatto GF. Phonemic fluency in Portuguese-speaking subjects in Brazil: ranking of letters. J Clin Exp Neuropsychol. 2006;28(7):1191-200. https://doi.org/10.1080/13803390500350969
https://doi.org/10.1080/1380339050035096...
,88. Steiner VAG, Mansur LL, Brucki SMD, Nitrini R. Phonemic verbal fluency and age: a preliminary study. Dement Neuropsychol. 2008;2(4):328-32. https://doi.org/10.1590/S1980-57642009DN20400017
https://doi.org/10.1590/S1980-57642009DN...
–1010. Rosselli M, Ardila A, Savatierra J, Marquez M, Matos L, Weekes VA. A cross-linguistic comparison of verbal fluency tests. Int J Neurosci. 2002;112(6):759-76. https://doi.org/10.1080/00207450290025752
https://doi.org/10.1080/0020745029002575...
They can be found in screening procedures or full language assessment batteries, including those for aphasia. Aphasia is caused by focal damage to cortical and/or subcortical structures of the dominant hemisphere(s) for verbal symbolic manipulations. It affects and is affected by other physiological information processes to the degree that they support, interact with, or are supported by symbolic deficits.1111. McNeil MR, Pratt S. Defining aphasia: Some theoretical and clinical implications of operating from a formal definition.Aphasiology. 2001;15(10-11):901-11. https://doi.org/10.1080/02687040143000276
https://doi.org/10.1080/0268704014300027...
Thus, VFTs are used for language assessment of people with aphasia (PWA).
Beyond the controversies about what exactly PVFs evaluate (language and/or executive functions), one point worth mentioning is the importance of analyzing strategies used during the phonological search. The task strategies are clustering and switching.1212. Troyer AK, Moscovitch M, Winocur G, Alexander MP, Stuss D. Clustering and switching on verbal fluency: the effects of focal frontal and temporal-lobe lesions. Neuropsychologia. 1998;36(6):499-504. https://doi.org/10.1016/s0028-3932(97)00152-8.
https://doi.org/10.1016/s0028-3932(97)00...
–1414. Troyer AK. Normative data for clustering and switching on verbal fluency task. J Clin Exp Neuropsychol. 2000;22(3):370-8. https://doi.org/10.1076/1380-3395(200006)22:3;1-V;FT370
https://doi.org/10.1076/1380-3395(200006...
Clustering is the production of groups of words according to meaning or sound. It depends on verbal memory processes related to the temporal lobe. When a given category ends, a switching occurs, which demands mental flexibility and is more closely related to the frontal lobe.1212. Troyer AK, Moscovitch M, Winocur G, Alexander MP, Stuss D. Clustering and switching on verbal fluency: the effects of focal frontal and temporal-lobe lesions. Neuropsychologia. 1998;36(6):499-504. https://doi.org/10.1016/s0028-3932(97)00152-8.
https://doi.org/10.1016/s0028-3932(97)00...
–1414. Troyer AK. Normative data for clustering and switching on verbal fluency task. J Clin Exp Neuropsychol. 2000;22(3):370-8. https://doi.org/10.1076/1380-3395(200006)22:3;1-V;FT370
https://doi.org/10.1076/1380-3395(200006...
The F-A-S test also allows the quantitative assessment of performance based on the total score (number of valid words produced), which is the most common use.22. Machado TH, Fichman HC, Santos EL, Carvalho VA, Fialho PP, Koenig AM, et al. Normative data for healthy elderly on the phonemic verbal fluency task-F-A-S. Dement Neuropsychol. 2009;3(1):55-60. https://doi.org/10.1590/S1980-57642009DN30100011
https://doi.org/10.1590/S1980-57642009DN...
,88. Steiner VAG, Mansur LL, Brucki SMD, Nitrini R. Phonemic verbal fluency and age: a preliminary study. Dement Neuropsychol. 2008;2(4):328-32. https://doi.org/10.1590/S1980-57642009DN20400017
https://doi.org/10.1590/S1980-57642009DN...
,99. Opasso PR, Barreto SS, Ortiz KZ. Fluência verbal fonêmica em adultos de alto letramento. Einstein (Sao Paulo). 2016;14(3):398-402. https://doi.org/10.1590/S1679-45082016AO3629
https://doi.org/10.1590/S1679-45082016AO...
Despite being a widely used procedure, few studies have investigated the performance of PWA on the PVF task specifically regarding the use of cognitive strategies.44. Whiteside DM, Kealey T, Semla M, Luu H, Rice L, Basso MR, et al. Verbal fluency: language or executive function measure? Appl Neuropsychol Adult. 2016;23(1):29-34. https://doi.org/10.1080/23279095.2015.1004574
https://doi.org/10.1080/23279095.2015.10...
,55. Aita SL, Beach JD, Taylo SE, Borgogna NC, Harrell MN, Hill BD. Executive, language, or both? An examination of the construct validity of verbal fluency measures. Appl Neuropsychol Adult. 2019;26(5):441-51. https://doi.org/10.1080/23279095.2018.1439830
https://doi.org/10.1080/23279095.2018.14...
,1515. Beckenkamp CL, Becker N, de Carvalho Rodrigues J, Fumagalli de Salles J. Estratégias de clustering and switching na evocação de adultos após acidente vascular cerebral nos hemisférios direito e esquerdo. Av Psicol Latinoam. 2019;37(2):375-90. http://dx.doi.org/10.12804/revistas.urosario.edu.co/apl/a.6073
http://dx.doi.org/10.12804/revistas.uros...
Recent investigations analyzing the use of these strategies in PVF tests did not involve PWA44. Whiteside DM, Kealey T, Semla M, Luu H, Rice L, Basso MR, et al. Verbal fluency: language or executive function measure? Appl Neuropsychol Adult. 2016;23(1):29-34. https://doi.org/10.1080/23279095.2015.1004574
https://doi.org/10.1080/23279095.2015.10...
,55. Aita SL, Beach JD, Taylo SE, Borgogna NC, Harrell MN, Hill BD. Executive, language, or both? An examination of the construct validity of verbal fluency measures. Appl Neuropsychol Adult. 2019;26(5):441-51. https://doi.org/10.1080/23279095.2018.1439830
https://doi.org/10.1080/23279095.2018.14...
or used diagnostic criteria that did not clearly differentiate aphasic from non-aphasic groups.1515. Beckenkamp CL, Becker N, de Carvalho Rodrigues J, Fumagalli de Salles J. Estratégias de clustering and switching na evocação de adultos após acidente vascular cerebral nos hemisférios direito e esquerdo. Av Psicol Latinoam. 2019;37(2):375-90. http://dx.doi.org/10.12804/revistas.urosario.edu.co/apl/a.6073
http://dx.doi.org/10.12804/revistas.uros...
The quantitative and qualitative analysis of the performance of aphasic individuals on the PVF test can contribute to a better understanding of cognitive changes in this group of patients. Besides, the PVF performance of PWA is possibly different and may not be extended to other populations referred to clinical neuropsychological assessments. One of the hypotheses of the present study is that the use of clustering and switching strategies might be associated with the number of words generated in the PVF F-A-S test by PWA.
How language and other cognitive aspects, such as attention and executive functions, interact1616. Peach RK, Beck KM, Gorman M, Fisher C. Clinical outcomes following language-specific attention treatment versus direct attention training for aphasia: a comparative effectiveness study. J Speech Lang Hear Res. 2019;62(8):2785-811. https://doi.org/10.1044/2019_JSLHR-L-18-0504
https://doi.org/10.1044/2019_JSLHR-L-18-...
during this test is another issue that still needs clarification. Therefore, we also investigated whether the qualitative analysis (based on clustering and switching strategies) is related to results from a non-verbal test of executive functions, that is, if PVF can really give us information about changes in cognitive domains other than language.
This study aimed to analyze clustering and switching measures of PVF in PWA and investigate the relationship between the use of these strategies, the quantitative performance on the test, and the performance on executive functions.
METHOD
This is an observational cross-sectional multicenter study. The study was approved by the Ethics Committee of both institutions (decisions no. 2,562,263 and 2,658,992). All patients signed informed consent forms prior to participation.
Participants
The participants were recruited from two speech therapy outpatient clinics of two Brazilian universities. The general inclusion criteria were: no previous history of neurological or psychiatric disorder, alcoholism, or drug use; no use of psychotropic medications, except for atypical neuroleptics; and absence of visual or auditory impairments as well as severe auditory comprehension disorder, as they could affect the test outcome. This study excluded patients with scores below the cut-off point for their schooling and age on the word and sentence oral comprehension subtests of the Brazilian version of the Montreal Toulouse Language Assessment Battery (MTL-BR).1717. Parente MA, Ortiz KZ, Soares EC, Scherer LC, Fonseca, RP, Joanette Y, et al. Bateria Montreal-Toulouse de Avaliação da Linguagem–Bateria MTL-Brasil. São Paulo: Vetor; 2014.,1818. Pagliarin KC, Ortiz KZ, Parente MA, Nespoulous JL, Joanette Y, Fonseca RP. Individual and sociocultural influences on language processing as assessed by the MTL-BR Battery. Aphasiology. 2014;28(10):1244-57. doi.org/10.1080/02687038.2014.918573 Monolingual adults and older people, native speakers of Brazilian Portuguese (BP), with a speech-language diagnosis of aphasia were included in this study. All patients were right-handed, had been evaluated by a neurologist, underwent neuroimaging tests, and were assessed with the MTL-BR for aphasia.
Fifteen patients were selected for this study. The group was very heterogeneous as to age (19 to 92 years old) and schooling (from 3 to 12 years of formal education). The prevalent etiology was stroke, and most PWA were at the chronic stage (from 2 to 115 months). Table 1 presents sociodemographic and clinical data.
Materials and Procedures
The F-A-S test was administered to assess PVF since it is the most studied version and enables a qualitative analysis of the speaker's performance.22. Machado TH, Fichman HC, Santos EL, Carvalho VA, Fialho PP, Koenig AM, et al. Normative data for healthy elderly on the phonemic verbal fluency task-F-A-S. Dement Neuropsychol. 2009;3(1):55-60. https://doi.org/10.1590/S1980-57642009DN30100011
https://doi.org/10.1590/S1980-57642009DN...
,66. Senhorini MC, Amaro Júnior E, Ayres AM, Simone A, Busatto GF. Phonemic fluency in Portuguese-speaking subjects in Brazil: ranking of letters. J Clin Exp Neuropsychol. 2006;28(7):1191-200. https://doi.org/10.1080/13803390500350969
https://doi.org/10.1080/1380339050035096...
,99. Opasso PR, Barreto SS, Ortiz KZ. Fluência verbal fonêmica em adultos de alto letramento. Einstein (Sao Paulo). 2016;14(3):398-402. https://doi.org/10.1590/S1679-45082016AO3629
https://doi.org/10.1590/S1679-45082016AO...
For the performance on the F-A-S test, the participants were asked to say as many words as possible, beginning with letters “F”, “A”, and “S” within a 1-minute period for each letter. The participants were instructed not to repeat items, say proper nouns or words derived from nominal or verbal inflections. An example with the letter “P” was provided by the examiner before the task. The letter P was selected due to its high frequency at the beginning of words in BP, similar to the incidence of letters F-A-S.88. Steiner VAG, Mansur LL, Brucki SMD, Nitrini R. Phonemic verbal fluency and age: a preliminary study. Dement Neuropsychol. 2008;2(4):328-32. https://doi.org/10.1590/S1980-57642009DN20400017
https://doi.org/10.1590/S1980-57642009DN...
The test complied with the criteria proposed by Senhorini et al.66. Senhorini MC, Amaro Júnior E, Ayres AM, Simone A, Busatto GF. Phonemic fluency in Portuguese-speaking subjects in Brazil: ranking of letters. J Clin Exp Neuropsychol. 2006;28(7):1191-200. https://doi.org/10.1080/13803390500350969
https://doi.org/10.1080/1380339050035096...
The procedure was recorded in audio format with a Moto Z Play 32GB mobile phone, model XT1635-02.
Answers were transcribed and analyzed according to the criteria proposed by Troyer et al.1212. Troyer AK, Moscovitch M, Winocur G, Alexander MP, Stuss D. Clustering and switching on verbal fluency: the effects of focal frontal and temporal-lobe lesions. Neuropsychologia. 1998;36(6):499-504. https://doi.org/10.1016/s0028-3932(97)00152-8.
https://doi.org/10.1016/s0028-3932(97)00...
,1313. Troyer AK, Moscovitch M, Winocur G. Clustering and switching as two components of verbal fluency: evidence from younger and older healthy adults. Neuropsychology. 1997;11(1):138-46. https://doi.org/10.1037//0894-4105.11.1.138
https://doi.org/10.1037//0894-4105.11.1....
Four scores were obtained: number of valid words (errors and repetitions were excluded), number of clusters, mean cluster size, and number of switches. All measures were calculated based on the total words retrieved in the three letters. Errors were computed in clustering and switching measures, since they also allow assessing the cognitive strategies used by the participants during their performance on the task. In this study, we also analyzed the number of clusters.
A clustering was considered when the participant produced: successive words generated with the same two initial letters (alliteration), word endings (rhymes) and/or words that differed only by a vowel, and homonyms (words with the same spelling but different meanings), as long as the participant was aware of the difference between them. A cluster was counted when it had at least two words. In cases of overlay of two subcategories with items belonging to both, with some belonging exclusively to one and others belonging exclusively to the other, the overlaid items were considered in both categories. However, whenever a small clustering was part of a bigger one, only the larger subcategory was considered. Cluster size was counted from the second word, and single words had a zero score in the calculation of mean cluster size. Switching, which corresponds to the number of transitions between the clustering, was counted even when isolated words were produced.1212. Troyer AK, Moscovitch M, Winocur G, Alexander MP, Stuss D. Clustering and switching on verbal fluency: the effects of focal frontal and temporal-lobe lesions. Neuropsychologia. 1998;36(6):499-504. https://doi.org/10.1016/s0028-3932(97)00152-8.
https://doi.org/10.1016/s0028-3932(97)00...
Besides clustering and switching measures, the type and frequency of other invalid words were also analyzed. Invalid words were classified into the following categories: intrusion (production initiated by different phoneme), perseveration (repetition of previous production, except valid words that were computed as repetition, according to the analysis proposed by Troyer et al.1212. Troyer AK, Moscovitch M, Winocur G, Alexander MP, Stuss D. Clustering and switching on verbal fluency: the effects of focal frontal and temporal-lobe lesions. Neuropsychologia. 1998;36(6):499-504. https://doi.org/10.1016/s0028-3932(97)00152-8.
https://doi.org/10.1016/s0028-3932(97)00...
,1313. Troyer AK, Moscovitch M, Winocur G. Clustering and switching as two components of verbal fluency: evidence from younger and older healthy adults. Neuropsychology. 1997;11(1):138-46. https://doi.org/10.1037//0894-4105.11.1.138
https://doi.org/10.1037//0894-4105.11.1....
), and neologism.
The Clock Drawing Test (CDT) investigated executive functions,1919. Sunderland T, Hill JL, Mellow AM, Lawlor BA, Gundersheimer J, Newhouse PA, et al. Clock drawing in Alzheimer's disease. J Am Geriatr Soc. 1989;37(8):725-9. https://doi.org/10.1111/j.1532-5415.1989.tb02233.x
https://doi.org/10.1111/j.1532-5415.1989...
–2121. Nishiwaki Y, Breeze E, Smeeth L, Bulpitt CJ, Peters J, Fletcher AE. Validity of the Clock-Drawing Test as a screening tool for cognitive impairment in the elderly. Am J Epidemiol. 2004;160(8):797-807. https://doi.org/10.1093/aje/kwh288
https://doi.org/10.1093/aje/kwh288...
which are also important in the performance of the F-A-S test. A non-verbal test was chosen due to the lack of validated verbal tests for executive function assessment in PWA, given their linguistic impairment. In addition, evidence shows that PWA perform worse on this test when compared to non-aphasic brain-injured patients.2222. Bonini MV, Radanovic M. Cognitive deficits in post-stroke aphasia. Arq Neuro-Psiquiatr. 2015;73(10):840-7. https://doi.org/10.1590/0004-282X20150133.
https://doi.org/10.1590/0004-282X2015013...
The cut-off point used for the study was 6.2323. Mendes-Santos LC, Mograbi D, Spenciere B, Charchat-Fichman H. Specific algorithm method of scoring the Clock Drawing Test applied in cognitively normal elderly. Dement Neuropsychol. 2015;9(2):128-35. https://doi.org/10.1590/1980-57642015DN92000007
https://doi.org/10.1590/1980-57642015DN9...
–2525. Noronha IF, Barreto SS, Ortiz KZ. The influence of education on performance of adults on the Clock Drawing Test. Dement Neuropsychol. 2018;12(1):61-7. https://doi.org/10.1590/1980-57642018dn12-010009
https://doi.org/10.1590/1980-57642018dn1...
In addition to the CDT, the Geriatric Depression Scale (GDS) was administered2626. Yesavage JA, Brink TL, Rose TL, Lum O, Huang V, Adey M, et al. Development and validation of a geriatric depression screening scale: a preliminary report. J Psychiatr Res. 1982-1983;17(1):37-49. https://doi.org/10.1016/0022-3956(82)90033-4
https://doi.org/10.1016/0022-3956(82)900...
The GDS was not adopted as an exclusion criterion because most PWA have depressive symptoms. Risk of depression was identified in 60% of the sample. However, subgroups with and without risk of depression were compared, and no differences were found between them on the F-A-S test measures.
Data were expressed as means of measures of central tendency and dispersion and relative frequency. For data analysis, we verified the non-adherence of numerical variables to the normal curve, using the Shapiro-Wilk test. The Spearman's test was applied to analyze the correlation among the F-A-S test scores and the CDT performance. Correlation strength was analyzed according to Siegel's2727. Siegel S. Estatística não paramétrica (para as ciências do comportamento). Porto Alegre: Bookman Companhia; 2006. proposal. The significance level adopted was p<0.05.
RESULTS
The mean PVF F-A-S test score of PWA was 5.9 words, with a standard deviation (SD) of 4.9. With respect to the cognitive strategies used by the participants, the mean scores for clustering (number and mean size) and switching were, respectively, 1.3 (SD=1.1), 0.9 (SD=2.5), and 1.9 (SD=2.3). Table 2 shows the results for each participant.
Only four participants produced errors, according to the Troyer et al.1212. Troyer AK, Moscovitch M, Winocur G, Alexander MP, Stuss D. Clustering and switching on verbal fluency: the effects of focal frontal and temporal-lobe lesions. Neuropsychologia. 1998;36(6):499-504. https://doi.org/10.1016/s0028-3932(97)00152-8.
https://doi.org/10.1016/s0028-3932(97)00...
criteria: four proper names, one derived word, and two repetitions. Eight participants generated 100 invalid words, of which 45% corresponded to intrusions, 28% to perseveration, and 27% to neologisms.
Table 3 presents the correlation among total score, clustering (number and mean size), and switching (number) measures in the F-A-S test. The total F-A-S test score showed a strong to moderate positive correlation with the clustering and switching measures analyzed.
Correlations among the scores for number of clusters, mean cluster size, and number of switches and the total scores for the F-A-S test and clock drawing test.
Regarding the CDT test, which assessed the performance of executive functions, the mean score was 4.9 (SD=3.1). Table 3 also presents the correlation between the F-A-S test and CDT measures. We found no correlation between the total score or the use of clustering and switching strategies and the CDT performance.
DISCUSSION
PWA performed poorly on the PVF test, both on the number of words retrieved and the cognitive strategies used to complete the task. Firstly, the mean number of words generated by the participants was considerably lower than the mean scores of previous studies carried out with healthy adult and older Brazilians (25.4 to 43.5).22. Machado TH, Fichman HC, Santos EL, Carvalho VA, Fialho PP, Koenig AM, et al. Normative data for healthy elderly on the phonemic verbal fluency task-F-A-S. Dement Neuropsychol. 2009;3(1):55-60. https://doi.org/10.1590/S1980-57642009DN30100011
https://doi.org/10.1590/S1980-57642009DN...
,99. Opasso PR, Barreto SS, Ortiz KZ. Fluência verbal fonêmica em adultos de alto letramento. Einstein (Sao Paulo). 2016;14(3):398-402. https://doi.org/10.1590/S1679-45082016AO3629
https://doi.org/10.1590/S1679-45082016AO...
,2828. Jardim AG. Emprego de estratégias de agrupamento e mudança de agrupamento no teste de fluência verbal fonêmica F-A-S por adultos saudáveis. Monografia (Graduação) – Universidade Federal Fluminense, Nova Friburgo, 2016. This result can be attributed to brain injury in the left hemisphere1515. Beckenkamp CL, Becker N, de Carvalho Rodrigues J, Fumagalli de Salles J. Estratégias de clustering and switching na evocação de adultos após acidente vascular cerebral nos hemisférios direito e esquerdo. Av Psicol Latinoam. 2019;37(2):375-90. http://dx.doi.org/10.12804/revistas.urosario.edu.co/apl/a.6073
http://dx.doi.org/10.12804/revistas.uros...
but mainly to the aphasic condition.2222. Bonini MV, Radanovic M. Cognitive deficits in post-stroke aphasia. Arq Neuro-Psiquiatr. 2015;73(10):840-7. https://doi.org/10.1590/0004-282X20150133.
https://doi.org/10.1590/0004-282X2015013...
In addition, the low educational level of the participants could also have interfered with the results (all participants, except one, had less than 11 years of formal education), as observed in a study of post-stroke patients with low schooling.1515. Beckenkamp CL, Becker N, de Carvalho Rodrigues J, Fumagalli de Salles J. Estratégias de clustering and switching na evocação de adultos após acidente vascular cerebral nos hemisférios direito e esquerdo. Av Psicol Latinoam. 2019;37(2):375-90. http://dx.doi.org/10.12804/revistas.urosario.edu.co/apl/a.6073
http://dx.doi.org/10.12804/revistas.uros...
The mean total number of words found in this study was similar to that of a previous study that investigated the performance of PWA on the F-A-S test, in which the mean and SD were 3.2 and 4.5.2222. Bonini MV, Radanovic M. Cognitive deficits in post-stroke aphasia. Arq Neuro-Psiquiatr. 2015;73(10):840-7. https://doi.org/10.1590/0004-282X20150133.
https://doi.org/10.1590/0004-282X2015013...
The reduced number of words generated in the PVF test by PWA might be associated with impaired access to the phonological lexicon, as processing occurs in the left brain hemisphere,1515. Beckenkamp CL, Becker N, de Carvalho Rodrigues J, Fumagalli de Salles J. Estratégias de clustering and switching na evocação de adultos após acidente vascular cerebral nos hemisférios direito e esquerdo. Av Psicol Latinoam. 2019;37(2):375-90. http://dx.doi.org/10.12804/revistas.urosario.edu.co/apl/a.6073
http://dx.doi.org/10.12804/revistas.uros...
often damaged in these patients.
Regarding the use of clustering and switching strategies, mean scores for number of switches were also lower when compared to previous studies of healthy subjects.1212. Troyer AK, Moscovitch M, Winocur G, Alexander MP, Stuss D. Clustering and switching on verbal fluency: the effects of focal frontal and temporal-lobe lesions. Neuropsychologia. 1998;36(6):499-504. https://doi.org/10.1016/s0028-3932(97)00152-8.
https://doi.org/10.1016/s0028-3932(97)00...
,1313. Troyer AK, Moscovitch M, Winocur G. Clustering and switching as two components of verbal fluency: evidence from younger and older healthy adults. Neuropsychology. 1997;11(1):138-46. https://doi.org/10.1037//0894-4105.11.1.138
https://doi.org/10.1037//0894-4105.11.1....
These studies presented the following scores: mean cluster size ranging from 0.4 to 0.51212. Troyer AK, Moscovitch M, Winocur G, Alexander MP, Stuss D. Clustering and switching on verbal fluency: the effects of focal frontal and temporal-lobe lesions. Neuropsychologia. 1998;36(6):499-504. https://doi.org/10.1016/s0028-3932(97)00152-8.
https://doi.org/10.1016/s0028-3932(97)00...
,1313. Troyer AK, Moscovitch M, Winocur G. Clustering and switching as two components of verbal fluency: evidence from younger and older healthy adults. Neuropsychology. 1997;11(1):138-46. https://doi.org/10.1037//0894-4105.11.1.138
https://doi.org/10.1037//0894-4105.11.1....
and number of switches, from 27.0 to 29.6.1212. Troyer AK, Moscovitch M, Winocur G, Alexander MP, Stuss D. Clustering and switching on verbal fluency: the effects of focal frontal and temporal-lobe lesions. Neuropsychologia. 1998;36(6):499-504. https://doi.org/10.1016/s0028-3932(97)00152-8.
https://doi.org/10.1016/s0028-3932(97)00...
,1313. Troyer AK, Moscovitch M, Winocur G. Clustering and switching as two components of verbal fluency: evidence from younger and older healthy adults. Neuropsychology. 1997;11(1):138-46. https://doi.org/10.1037//0894-4105.11.1.138
https://doi.org/10.1037//0894-4105.11.1....
Studies analyzing the number of clusters in the F-A-S test of adults were not found. The worst performance of PWA on number of switches suggests that, in addition to linguistic impairment (access to the phonological lexicon), the use of cognitive flexibility to search for new strategies to perform the task might also be compromised.
The clustering and switching strategies used by PWA had the same pattern observed in studies with PVF tests in this population,1515. Beckenkamp CL, Becker N, de Carvalho Rodrigues J, Fumagalli de Salles J. Estratégias de clustering and switching na evocação de adultos após acidente vascular cerebral nos hemisférios direito e esquerdo. Av Psicol Latinoam. 2019;37(2):375-90. http://dx.doi.org/10.12804/revistas.urosario.edu.co/apl/a.6073
http://dx.doi.org/10.12804/revistas.uros...
especially for switching measures. Evidence shows that right hemisphere post-stroke patients did not present this pattern in PVF tests. In these cases, the number of switches was comparable to that of neurologically healthy people.1515. Beckenkamp CL, Becker N, de Carvalho Rodrigues J, Fumagalli de Salles J. Estratégias de clustering and switching na evocação de adultos após acidente vascular cerebral nos hemisférios direito e esquerdo. Av Psicol Latinoam. 2019;37(2):375-90. http://dx.doi.org/10.12804/revistas.urosario.edu.co/apl/a.6073
http://dx.doi.org/10.12804/revistas.uros...
This evidence reinforces the hypothesis that the reduced number of switches produced by PWA might be mainly associated with deficits in the specific linguistic processing that occurs in the left hemisphere since deficits in executive functions are not exclusive to this hemisphere.
The present study participants produced more invalid words than errors or repetitions considering the criteria proposed by Troyer et al.1212. Troyer AK, Moscovitch M, Winocur G, Alexander MP, Stuss D. Clustering and switching on verbal fluency: the effects of focal frontal and temporal-lobe lesions. Neuropsychologia. 1998;36(6):499-504. https://doi.org/10.1016/s0028-3932(97)00152-8.
https://doi.org/10.1016/s0028-3932(97)00...
Regarding the invalid words, intrusions were the most common, followed by perseverations and neologisms. The occurrence of intrusions and perseverations could be related to inhibitory control failures. Although neologisms are a linguistic manifestation of the aphasic condition, their occurrence suggests phonological disintegration. Future analysis including other frequent types of language errors made by PWA can contribute to understanding the cognitive strategies used by them during the VFT.
Despite the lower PVF scores identified in PWA, strong to moderate positive correlations were found among the scores for number of clusters, mean cluster size, and number of switches and total F-A-S test score, as shown in Table 3. The total score in the test shows a strong correlation with the number of switches, as observed in healthy people.1212. Troyer AK, Moscovitch M, Winocur G, Alexander MP, Stuss D. Clustering and switching on verbal fluency: the effects of focal frontal and temporal-lobe lesions. Neuropsychologia. 1998;36(6):499-504. https://doi.org/10.1016/s0028-3932(97)00152-8.
https://doi.org/10.1016/s0028-3932(97)00...
In PVF tasks, switching was the most used strategy by people without left hemisphere brain damage.1515. Beckenkamp CL, Becker N, de Carvalho Rodrigues J, Fumagalli de Salles J. Estratégias de clustering and switching na evocação de adultos após acidente vascular cerebral nos hemisférios direito e esquerdo. Av Psicol Latinoam. 2019;37(2):375-90. http://dx.doi.org/10.12804/revistas.urosario.edu.co/apl/a.6073
http://dx.doi.org/10.12804/revistas.uros...
According to Shao et al.,2929. Shao Z, Janse E, Visser K, Meyer AS. What do verbal fluency tasks measure? Predictors of verbal fluency performance in older adults. Front Psychol. 2014;5:772. https://doi.org/10.3389/fpsyg.2014.00772.eCollection2014
https://doi.org/10.3389/fpsyg.2014.00772...
associations with words with the same initial letter are less frequent, so their access is more difficult. Namely, the reason for varying switching strategies is more effective in a PVF task. In the present study, the strong positive correlation found between the number of switches and the total number of words generated indicates that PWA use this cognitive strategy less often, leading to a smaller number of retrieved words.
Considering that switching is associated with mental flexibility, we expected to find a relationship between switching and CDT. However, such correlation was not detected (Table 3). This result reinforces that the use of cognitive strategies in the F-A-S test (including the number of switches) may not involve the executive functions assessed by the CDT. In fact, the debate about the relationship between language and other cognitive domains is not new. More recently, some authors have argued that the cognition involved in language processing is domain specific,1111. McNeil MR, Pratt S. Defining aphasia: Some theoretical and clinical implications of operating from a formal definition.Aphasiology. 2001;15(10-11):901-11. https://doi.org/10.1080/02687040143000276
https://doi.org/10.1080/0268704014300027...
that is, each cognitive domain is allocated in ways that are particular to specific tasks rather than as a general resource equivalently allocated to all processing tasks.1616. Peach RK, Beck KM, Gorman M, Fisher C. Clinical outcomes following language-specific attention treatment versus direct attention training for aphasia: a comparative effectiveness study. J Speech Lang Hear Res. 2019;62(8):2785-811. https://doi.org/10.1044/2019_JSLHR-L-18-0504
https://doi.org/10.1044/2019_JSLHR-L-18-...
This hypothesis suggests that choosing a non-verbal test for this purpose would not be appropriate.
Regarding the possible association between the use of clustering and switching and the integrity of the frontal and temporal lobes, respectively, the reduced sample and the heterogeneity of the brain lesions limited a specific analysis of this aspect. Five patients had a frontal lobe lesion, associated or not with a lesion in another brain area except the left temporal lobe (P03, P04, P06, P11, P14), and only one patient presented a lesion in the left temporal lobe without damage to the frontal lobe (P08). Scores for the number of clusters and mean cluster size from P08 did not differ from means of these measures in the subgroup with damage to the frontal lobe, and the number of switches was lower than the mean observed in these subgroups (0 versus 1.2). Therefore, we found no trend toward worse switching performance in PWA with injuries involving the frontal lobe, as observed by Troyer et al.1313. Troyer AK, Moscovitch M, Winocur G. Clustering and switching as two components of verbal fluency: evidence from younger and older healthy adults. Neuropsychology. 1997;11(1):138-46. https://doi.org/10.1037//0894-4105.11.1.138
https://doi.org/10.1037//0894-4105.11.1....
This study evaluated a small number of PWA with different clinical conditions and sociodemographic characteristics. These aspects and the lack of other clinical groups or normal subjects limited our data interpretation. Nevertheless, the patients presented different language disorders (different types of aphasia) and were assessed by a standardized language battery specific to this population.
The quantitative and qualitative analysis of F-A-S in PWA, even those with different linguistic manifestations, showed that these individuals presented lower scores and that the total number of words and the number of switches were strongly correlated. The present study found no correlation between executive functions, assessed by the CDT, and switching performance on the F-A-S test. Such results suggest that the PVF performance by PWA could not be explained only by an executive impairment, but mainly by the linguistic deficit. The switching strategy analysis, coupled with the analysis of other errors commonly found in PWA, can contribute to a better understanding of the phonological-lexical access difficulties in this population, as well as the use of other cognitive strategies to perform the task, with possible implications for the rehabilitation process.
-
This study was conducted at the Department of Specific Training in Speech-Language Pathology, Nova Friburgo Health Institute, Universidade Federal Fluminense, Nova Friburgo, RJ, Brazil.
-
Funding: Fundação Carlos Chagas Filho de Amparo à Pesquisa do Estado do Rio de Janeiro (FAPERJ) — E-26/ 200.346/2018 process.
ACKNOWLEDGEMENTS
We thank Yasmin da Silva Souza and Carla de Oliveira Pinto for their contribution to data collection.
REFERENCES
-
1Ruff RM, Light RH, Parker SB, Levin HS. The psychological construct of word fluency. Brain Lang. 1997;57(3):394-405. https://doi.org/10.1006/brln.1997.1755
» https://doi.org/10.1006/brln.1997.1755 -
2Machado TH, Fichman HC, Santos EL, Carvalho VA, Fialho PP, Koenig AM, et al. Normative data for healthy elderly on the phonemic verbal fluency task-F-A-S. Dement Neuropsychol. 2009;3(1):55-60. https://doi.org/10.1590/S1980-57642009DN30100011
» https://doi.org/10.1590/S1980-57642009DN30100011 -
3Lezak MD, Howieson DB, Biegler ED, Tranel D. Neuropsychological Assessment. 2. ed. New York: Oxford University Press; 2012.
-
4Whiteside DM, Kealey T, Semla M, Luu H, Rice L, Basso MR, et al. Verbal fluency: language or executive function measure? Appl Neuropsychol Adult. 2016;23(1):29-34. https://doi.org/10.1080/23279095.2015.1004574
» https://doi.org/10.1080/23279095.2015.1004574 -
5Aita SL, Beach JD, Taylo SE, Borgogna NC, Harrell MN, Hill BD. Executive, language, or both? An examination of the construct validity of verbal fluency measures. Appl Neuropsychol Adult. 2019;26(5):441-51. https://doi.org/10.1080/23279095.2018.1439830
» https://doi.org/10.1080/23279095.2018.1439830 -
6Senhorini MC, Amaro Júnior E, Ayres AM, Simone A, Busatto GF. Phonemic fluency in Portuguese-speaking subjects in Brazil: ranking of letters. J Clin Exp Neuropsychol. 2006;28(7):1191-200. https://doi.org/10.1080/13803390500350969
» https://doi.org/10.1080/13803390500350969 -
7Casals-Coll M, Sánchez-Benavides G, Quintana M, Manero RM, Rognoni T, Calvo L, et al. Estudios normativos españoles em población adulta joven (proyecto NEURONORMA jóvenes): normas para lós test de fluencia verbal. Neurología. 2013;28(1):33-40. https://doi.org/10.1016/j.nrl.2012.02.010
» https://doi.org/10.1016/j.nrl.2012.02.010 -
8Steiner VAG, Mansur LL, Brucki SMD, Nitrini R. Phonemic verbal fluency and age: a preliminary study. Dement Neuropsychol. 2008;2(4):328-32. https://doi.org/10.1590/S1980-57642009DN20400017
» https://doi.org/10.1590/S1980-57642009DN20400017 -
9Opasso PR, Barreto SS, Ortiz KZ. Fluência verbal fonêmica em adultos de alto letramento. Einstein (Sao Paulo). 2016;14(3):398-402. https://doi.org/10.1590/S1679-45082016AO3629
» https://doi.org/10.1590/S1679-45082016AO3629 -
10Rosselli M, Ardila A, Savatierra J, Marquez M, Matos L, Weekes VA. A cross-linguistic comparison of verbal fluency tests. Int J Neurosci. 2002;112(6):759-76. https://doi.org/10.1080/00207450290025752
» https://doi.org/10.1080/00207450290025752 -
11McNeil MR, Pratt S. Defining aphasia: Some theoretical and clinical implications of operating from a formal definition.Aphasiology. 2001;15(10-11):901-11. https://doi.org/10.1080/02687040143000276
» https://doi.org/10.1080/02687040143000276 -
12Troyer AK, Moscovitch M, Winocur G, Alexander MP, Stuss D. Clustering and switching on verbal fluency: the effects of focal frontal and temporal-lobe lesions. Neuropsychologia. 1998;36(6):499-504. https://doi.org/10.1016/s0028-3932(97)00152-8
» https://doi.org/10.1016/s0028-3932(97)00152-8 -
13Troyer AK, Moscovitch M, Winocur G. Clustering and switching as two components of verbal fluency: evidence from younger and older healthy adults. Neuropsychology. 1997;11(1):138-46. https://doi.org/10.1037//0894-4105.11.1.138
» https://doi.org/10.1037//0894-4105.11.1.138 -
14Troyer AK. Normative data for clustering and switching on verbal fluency task. J Clin Exp Neuropsychol. 2000;22(3):370-8. https://doi.org/10.1076/1380-3395(200006)22:3;1-V;FT370
» https://doi.org/10.1076/1380-3395(200006)22:3;1-V;FT370 -
15Beckenkamp CL, Becker N, de Carvalho Rodrigues J, Fumagalli de Salles J. Estratégias de clustering and switching na evocação de adultos após acidente vascular cerebral nos hemisférios direito e esquerdo. Av Psicol Latinoam. 2019;37(2):375-90. http://dx.doi.org/10.12804/revistas.urosario.edu.co/apl/a.6073
» http://dx.doi.org/10.12804/revistas.urosario.edu.co/apl/a.6073 -
16Peach RK, Beck KM, Gorman M, Fisher C. Clinical outcomes following language-specific attention treatment versus direct attention training for aphasia: a comparative effectiveness study. J Speech Lang Hear Res. 2019;62(8):2785-811. https://doi.org/10.1044/2019_JSLHR-L-18-0504
» https://doi.org/10.1044/2019_JSLHR-L-18-0504 -
17Parente MA, Ortiz KZ, Soares EC, Scherer LC, Fonseca, RP, Joanette Y, et al. Bateria Montreal-Toulouse de Avaliação da Linguagem–Bateria MTL-Brasil. São Paulo: Vetor; 2014.
-
18Pagliarin KC, Ortiz KZ, Parente MA, Nespoulous JL, Joanette Y, Fonseca RP. Individual and sociocultural influences on language processing as assessed by the MTL-BR Battery. Aphasiology. 2014;28(10):1244-57. doi.org/10.1080/02687038.2014.918573
-
19Sunderland T, Hill JL, Mellow AM, Lawlor BA, Gundersheimer J, Newhouse PA, et al. Clock drawing in Alzheimer's disease. J Am Geriatr Soc. 1989;37(8):725-9. https://doi.org/10.1111/j.1532-5415.1989.tb02233.x
» https://doi.org/10.1111/j.1532-5415.1989.tb02233.x -
20Manos PJ. The utility of the ten-point clock test as a screen for cognitive impairment in general hospital patients. Gen Hosp Psychiatry. 1997;19(6):439-44. https://doi.org/10.1016/s0163-8343(97)00072-8
» https://doi.org/10.1016/s0163-8343(97)00072-8 -
21Nishiwaki Y, Breeze E, Smeeth L, Bulpitt CJ, Peters J, Fletcher AE. Validity of the Clock-Drawing Test as a screening tool for cognitive impairment in the elderly. Am J Epidemiol. 2004;160(8):797-807. https://doi.org/10.1093/aje/kwh288
» https://doi.org/10.1093/aje/kwh288 -
22Bonini MV, Radanovic M. Cognitive deficits in post-stroke aphasia. Arq Neuro-Psiquiatr. 2015;73(10):840-7. https://doi.org/10.1590/0004-282X20150133
» https://doi.org/10.1590/0004-282X20150133 -
23Mendes-Santos LC, Mograbi D, Spenciere B, Charchat-Fichman H. Specific algorithm method of scoring the Clock Drawing Test applied in cognitively normal elderly. Dement Neuropsychol. 2015;9(2):128-35. https://doi.org/10.1590/1980-57642015DN92000007
» https://doi.org/10.1590/1980-57642015DN92000007 -
24Fabricio AT, Aprahamian I, Yassuda MS. Qualitative analysis of the Clock Drawing Test by educational level and cognitive profile. Arq Neuro-Psiquiatr. 2014;72(4):289-95. https://doi.org/10.1590/0004-282x20140004
» https://doi.org/10.1590/0004-282x20140004 -
25Noronha IF, Barreto SS, Ortiz KZ. The influence of education on performance of adults on the Clock Drawing Test. Dement Neuropsychol. 2018;12(1):61-7. https://doi.org/10.1590/1980-57642018dn12-010009
» https://doi.org/10.1590/1980-57642018dn12-010009 -
26Yesavage JA, Brink TL, Rose TL, Lum O, Huang V, Adey M, et al. Development and validation of a geriatric depression screening scale: a preliminary report. J Psychiatr Res. 1982-1983;17(1):37-49. https://doi.org/10.1016/0022-3956(82)90033-4
» https://doi.org/10.1016/0022-3956(82)90033-4 -
27Siegel S. Estatística não paramétrica (para as ciências do comportamento). Porto Alegre: Bookman Companhia; 2006.
-
28Jardim AG. Emprego de estratégias de agrupamento e mudança de agrupamento no teste de fluência verbal fonêmica F-A-S por adultos saudáveis. Monografia (Graduação) – Universidade Federal Fluminense, Nova Friburgo, 2016.
-
29Shao Z, Janse E, Visser K, Meyer AS. What do verbal fluency tasks measure? Predictors of verbal fluency performance in older adults. Front Psychol. 2014;5:772. https://doi.org/10.3389/fpsyg.2014.00772.eCollection2014
» https://doi.org/10.3389/fpsyg.2014.00772.eCollection2014
Publication Dates
-
Publication in this collection
11 Dec 2020 -
Date of issue
Dec 2020
History
-
Received
14 Feb 2019 -
Accepted
01 Sept 2020