The Brazilian version of the Constant-Murley Score (CMS-BR): convergent and construct validity, internal consistency, and unidimensionality^☆

ABSTRACT

OBJECTIVES:

To translate and culturally adapt the CMS and assess the validity of the Brazilian version (CMS-BR).

METHODS:

The translation was carried out according to the back-translation method by four independent translators. The produced versions were synthesized through extensive analysis and by consensus of an expert committee, reaching a final version used for the cultural adaptation. A field test was conducted with 30 subjects in order to obtain semantic considerations. For the psychometric analyzes, the sample was increased to 110 participants who answered two instruments: CMS-BR and the Disabilities of the Arm, shoulder and Hand (DASH). The CMS-BR and DASH score range from 0 to 100 points. For the first, higher points reflect better function and for the latter, the inverse is true. The validity was verified by Pearson's correlation test, the unidimensionality by factorial analysis, and the internal consistency by Cronbach's alpha.

RESULTS:

The explained variance was 60.28% with factor loadings ranging from 0.60 to 0.91. The CMS-BR exhibited strong negative correlation with the DASH score (-0.82, p < 0.05), Cronbach's alpha 0.85, and its total score was strongly correlated with the patient's range of motion (0.93, p < 0.001).

CONCLUSION:

The CMS was satisfactorily adapted for Brazilian Portuguese and demonstrated evidence of validity that allows its use in this population.

Keywords:
Clinimetrics; Assessment; Factor analysis; Validity; Shoulder

RESUMO

OBJETIVOS:

Traduzir e adaptar culturalmente o Constant-Murley Score (CMS) e verificar a validade da versão brasileira (CMS-BR).

MÉTODOS:

A tradução foi realizada de acordo com o método de retrotradução por quatro tradutores independentes. As versões produzidas foram sintetizadas por análise extensiva e consenso de um comitê de especialistas, gerando uma versão final usada para a adaptação cultural. Realizou-se um teste em campo com 30 sujeitos para observação de possíveis considerações em relação à semântica. Para a posterior análise psicométrica, ampliou-se a amostra para 110 participantes, que responderam a dois instrumentos: CMS-BR e Disabilities of the Arm, Shoulder and Hand (DASH). O CMS-BR e o DASH possuem variação de 0 a 100 pontos. Para o primeiro, altas pontuações refletem melhor função, para o segundo, o contrário. A validade foi verificada através do teste de correlação de Pearson, a dimensionalidade através de análise fatorial e a consistência interna através do alfa de Cronbach.

RESULTADOS:

A variância explicada foi de 60,28% com cargas fatoriais entre 0,60 e 0,91. O CMS-BR demonstrou correlação forte e negativa com o DASH (-0,82, p < 0,05), alfa de Cronbach de 0,85 e seu escore total teve correlação forte com a amplitude de movimento dos pacientes (0,93, p < 0,001).

CONCLUSÃO:

O CMS-BR foi adaptado de forma satisfatória e demonstrou evidências de validade que permitem seu uso nessa população.

Palavras-chave:
Clinimetria; Avaliação; Análise fatorial; Validade; Ombro

Introduction

Shoulder pain accounts for an expressive prevalence in consultations with a general practitioner or orthopedic surgeons.^1and2 These patients often present various complaints, like mobility deficits and pain,³ which directly affect upper limb function. In order to perform an as comprehensive clinical assessment as possible is recommended that patient be assessed with instruments that allow inferences about function. The function is a construct, a latent variable that cannot be directly observed. Therefore, the utilization of functional scores is the adequate option to measure it.^4and5

There are about 34 scores for shoulder function assessment but the Constant-Murley score (CMS), originally published in the English language, is one of the most used.^{4, 6-7} The CMS is a non-specific score that covers different domains of shoulder function (pain, activities of daily living, range of motion and power) being higher scores indicative of better function.^6,7and8 This instrument is a compound score containing four subscales: three self-reported subscales and one shoulder elevation strength subscale which is performed by an external assessor.⁸ The nomenclature of the "power" subscale contained in the original version of the CMS was posteriorly changed to "strength", as well as the test position was changed to elevation in scapular plane.⁹

The appropriate use of an instrument of evaluation implies the correct verification of its validity.^10and11 The evidence of validity characterize the relationship among items of the score and between items and total score. It also indicates the extent in which the instrument explains the construct under assessment. This process ensures an adequate representation of the construct measured by the functional score.^12and13

Psychometric properties of the original version of the CMS such as reliability, floor and ceiling effects, convergent and criterion validity have been verified. Despite the comprehensive investigation of the validity of the score, its dimensional structure was investigated before the adaptation of the strength subscale and the factor analysis evinced that the score was not unidimensional.¹⁴ These aforementioned features could affect the interpretation of measurement of the construct.^5,12and15

The use of an instrument of evaluation in another culture or language must be preceded by an appropriate process of translation and cultural adaptation. Furthermore, evidence of validity must be properly verified in the adapted version. Currently, a translated and adapted version of CMS is available only for the Danish^16and17 language. There is no version of CMS in the Brazilian Portuguese language. Therefore, the aim of this study was translate, culturally adapt and verify the convergent and construct validity, internal consistency and dimensional structure of the adapted version.

Methods

The process of translation was performed according the back-translation method^10and11 and following the COSMIN checklist for ensure the methodological quality of the psychometric analysis.¹⁸

The recommendations published by Constant et al.⁹ were followed, excepting for the branding and model of the dynamometer. In this study an isometric dynamometer model MicroFET2(tm) (Hoggan Health Industries, USA) with a sensitivity of 0.05 kg (0.1 lb), measuring up to 136.05 kg (300 lb) was utilized in the strength subscale.

Translation and cultural adaption

The process of translation occurred in four steps: translation, back translation, analysis of preliminary version and field test. Initially, two expert and independent Brazilian Portuguese native language translators provided two translated versions of the original CMS. These versions were unified for a consensus between authors and translators.

Afterward, the unified translated version was back-translated by two expert and independent English native language translators. Both translators had no access to the original CMS. These versions were unified by consensus of the authors group. The next step involved the establishment of a fully culturally adapted version taking account all existent versions obtained from the previous steps through a multidisciplinary experts committee meeting. Finally, in order to collect semantic considerations, the Brazilian Language version of CMS (CMS-BR) was applied in a sample of 30 subjects with similar characteristics of the main study.

Participants

The participants were selected from private practice clinics of Porto Alegre and Novo Hamburgo, Brazil. This study received approval from the Research Ethics Committee of the proponent institution (study number 992-12) and all participants signed written informed consent prior to the enrollment.

A hundred and ten patients (55 male) older than 18 years old, with any diagnoses of clinical shoulder dysfunction (except instability) and able to read and answer to the questionnaire were included. Patients with cognitive impairments, peripheral or central nerve damage or diagnose of nerve dysfunction were excluded of the study. The mean age of the included individuals was 48.50 (15.13) and ranged between 18 and 83 years.

Statistical analysis

A descriptive analysis of all variables was performed. Categorical data were expressed as absolute or relative frequencies and quantitative variables as means and standard deviations.

Clinimetric analysis

Evidence of validity was analyzed through the following statements^5,12,18and19:

For internal consistency, Cronbach's alpha test considered the value >0.80 as ideal.^20and21 The convergent validity was verified through Pearson's correlation between the total scores of the CMS-BR and the Brazilian version of the Disabilities of the arm, shoulder and hand score (DASH)²² adopting a r ≥ 0.70 and a p ≤ 0.05 to satisfy this condition. ^12and18 The construct validity was verified by the Pearson correlation test between the range of motion of all subjects and the CMS-BR final score. The hypothesis was that the poor functional status would be associated with a less active range of motion in the assessed shoulder.⁵

Since the CMS-BR presupposes the assessment of only one construct (shoulder function) and one factor extraction was indicated as the better solution through scree plot and the total explained variance (eigenvalue), a factor analysis with an exploratory principal component analysis with a one-factor solution was performed to verify the dimensional structure. Finally, this analysis considered some assumptions and related tests previous to its realization^15and19:

• The Kaiser-Meyer-Olkin (KMO) coefficient must be >0.70;

• The determinant of the correlation matrix must not be zero, but a value closest to zero;

• Barlett's sphericity test must have a p ≤ 0.05;

• The communality value must be ≥0.4.^23and24

Results

The list of conditions of all included individuals can be observed in Table 1. The CMS-BR was obtained through adequate translation process following recommendations of the literature. Only one modification was made in the activities of daily living subscale to improve the understanding of the CMS-BR. Specifically, in the last question about which is the arm elevation level without pain, the expression "up to xiphoid" was replaced by "up to heart level" ("ao nível do coração" in Brazilian Portuguese) as can be observed in ^{Appendix 1}.

Table 1
List of conditions.

Conditions	Absolute frequency	Relative frequency
OA AC	4	3.64%
OA GH	1	0.91%
Adhesive capsulitis	15	13.64%
SIS	26	23.64%
Proximal fracture of humerus	9	8.18%
ACD	2	1.82%
RCT	22	20.00%
Bursectomy plus Acromioplasty	2	1.82%
Rotator cuff repair	16	14.55%
Calcific tendinitis	3	2.73%
Suprascapular nerve release	1	0.91%
Healthy	9	8.18%
Total	110	100%

• AC, acromioclavicular joint; OA, osteoarthritis; GH, glenohumeral joint; SIS, subacromial impingement syndrome; ACD, acromioclavicular joint dislocation; RCT, rotator cuff tear.

Regarding internal consistency, a moderate to strong correlation was observed in each item-total correlation. The mean CMS-BR score was 49.69 (28.12) and the Cronbach's alpha was 0.85. The alpha did not significantly increase with the hypothetical exclusion of any item, confirming the score arrangement of items. Moreover, there was no floor and ceiling effects, when more than 15% of the respondents achieves the highest or lowest possible scores (Table 2).

Table 2
Item-total statistics.

Item	Min	Max	Mean (SD)	Item-total correlation	Alpha if item deleted
Pain	0	15	8.46 (4.37)	0.49	0.84
Sleep	0	2	0.88 (0.73)	0.63	0.85
Work	0	4	2.56 (1.42)	0.57	0.85
Leisure	0	4	2.26 (1.45)	0.61	0.84
Level of elevation	2	10	5.98 (2.84)	0.80	0.82
Flexion	0	10	5.24 (3.50)	0.83	0.81
Abduction	0	10	4.91 (3.36)	0.86	0.81
External rotation	0	10	5.53 (3.81)	0.74	0.82
Internal rotation	0	10	5.55 (3.61)	0.69	0.82
Strength	0	35	8.32 (10.04)	0.75	0.88

• SD, standard deviation. Recommended Cronbach's Alpha: ≥0.80.

The CMS-BR and Brazilian DASH presented a significantly strong negative correlation (r = -0.82, p ≤ 0.001). The direction of this association is due DASH scoring system which higher score is indicative of poor shoulder function conversely to CMS-BR. Moreover, the CMS-BR and the range of motion of the participants were strongly related (r = 0.93, p ≤ 0.001), confirming the a priori hypothesis that poor functional status is related to less active range of motion.

The dimensional structure of the CMS-BR was tested by factor analysis with an exploratory principal component analysis with one-factor solution. All assumptions for the test realization were fulfilled. The explained variance by one-factor extraction was 60.28%. The communalities ranged from 0.36 (pain) to 0.83 (Table 3).

Table 3
Communalities and factorial loads of items.

Item	Communalities	Factorial load
Abduction	0.83	0.91
Flexion	0.80	0.89
Level of elevation	0.74	0.86
Strength	0.65	0.80
External rotation	0.65	0.80
Internal rotation	0.57	0.75
Leisure	0.49	0.70
Sleep	0.47	0.68
Work	0.44	0.66
Pain	0.36	0.60

• Recommended value for communality: ≥0.40.

Discussion

This was the first study to translate and culturally adapt the CMS to Brazilian Portuguese. The preliminary version of CMS-BR was extensively analyzed by physicians, orthopedic-trauma residents, physical therapists, a nurse and a statistician, comprising nine professionals with different backgrounds and was applied in 30 patients. Moreover, important evidences of validity were tested in the CMS-BR such as internal consistency, convergent validity and dimensional structure.

The analysis showed that the CMS-BR presented a high internal consistency (Cronbach's alpha = 0.85). The alpha did not increase with the hypothetical exclusion of any item confirming the layout of the adapted score. Interestingly, the internal consistency of the original CMS ranged from 0.60 to 0.75.^25and26 A systematic review suggested that the low alpha values may indicate that the CMS items measure different aspects of shoulder function.²⁷ To date, there is no objective data to explain the marked observed differences in the alpha values between the CMS and CMS-BR. However, the modifications proposed in 2008⁹ might play a role on it. Furthermore, a moderate to strong correlation was observed in each item-total correlation (Table 2).

Regarding convergent validity, the CMS-BR demonstrated a strong negative correlation (r = -0.82, p ≤ 0.001) with the Brazilian version of the DASH score, although the CMS presented a low to moderate association with DASH.²⁷ The construct validity was confirmed through the significantly strong correlation (r = 0.93, p ≤ 0.001) between CMS-BR score and range of motion. The hypothesis raised by the authors seems totally appropriated since the range of motion is an important characteristic for the shoulder joint and frequently impaired in the majority of shoulder dysfunctions. The Danish version of the CMS also demonstrated a strong correlation with the Oxford Shoulder Score (r = 0.76). In spite of the similar values found in both adapted versions, our reference standard score was the Brazilian DASH - a widely used in myriad of shoulder conditions - while the Oxford Shoulder Score is more suited to assess surgical populations and proximal humerus fractures. ^6and28

The factor analysis of the CMS-BR evinced that the amount of variance explained by one-factor solution was 60.28%, ensuring that the CMS-BR met the one-dimensionality criterion.¹⁵ In the original CMS, the factor analysis with a two-factor solution was performed by only one study.¹⁴ However, the authors did not report neither the factor loads nor the adopted criterion to analysis. According to our impression, possibly the lack of standardization, mainly in the pain or strength subscales, could justify these discrepancies in the dimensional structure. A standardized process of implementation was adopted for the realization of our study further the recommendations aforementioned.

Other psychometric properties were published about the original CMS. Floor and ceiling effects were analyzed and the strength subscale reported a considerable floor effect.²⁹ Many patients were unable to hold the proper position for strength assessment receiving zero points.²⁹ The same report was observed in another study that assessed patients with adhesive capsulitis.³⁰ Although the strength subscale account for 25% of total score, this not seemed to interfere in the reliability for pain and strength subscales.^25and31 Recently, the minimally important change was verified to patients with subacromial pain.³²

Even that in the present study some properties have not been analyzed, the CMS-BR have satisfactory tested four important attributes of validity and figure among the Brazilian adapted scores with more psychometrics verifications. Puga et al.³³ performed a systematic review which analyzed psychometric properties of all published scores adapted to Brazilian Portuguese until 2011. Of concern is the fact that all included studies in this review did not analyze more than one attribute of validity, which hampers clinical and research usefulness of these instruments.

Conversely, the psychometric properties of the Brazilian version of the Penn Shoulder Score were recently verified.³⁴ The authors reported a comprehensive analysis of the psychometric properties of this instrument, such as internal consistency, measurement error, construct validity and floor and ceiling effects. Furthermore, Moser et al.³⁵ analyzed internal consistency, convergent validity and reliability of the Brazilian version of the American Shoulder and Elbow Surgeons (ASES) score. Adequate results that support the utilization of both scores were observed.

Neto et al.³⁶ performed the translation and cultural adaptation of the Simple Shoulder Test for Brazilian Portuguese (SST-BR). However, some methodological issues must be considered. In the preliminary version analysis, the expert committee was composed for one physician and six translators. Actually, current recommendations suggest that the expert committee must be composed by different professionals, in order to provide an as complete as possible adapted version to the target population.^10and11 The authors also performed the dimensional structure verification of the SST-BR. However, an exploratory and a subsequent confirmatory factor analysis were performed with a three-factor solution in opposition to the original measurement concept of the score.

Our study exhibits some limitations. The verification of reproducibility (agreement and reliability), responsiveness, minimum detectable change and minimally important change were not performed. Nonetheless, firstly is important ensure that the score really measures the target construct (function), and after to analyze more validity properties.

Conclusion

From the results aforementioned it was evinced that the CMS-BR was satisfactory adapted to Brazilian Portuguese culture. Moreover, the CMS-BR has adequate convergent and construct validity, internal consistency and adequate dimensional structure that support its utilization in clinical practice for the evaluation of patients with shoulder dysfunctions.

Acknowledgments

We would like to thank PT Giovanni Ferreira and MSc. Caroline Robinson for the English language review. We also thank PT Carlos Vicente da Silva for the artistic adjustments in the preliminary version of the score. We would like to sincerely thank MD Fábio Matsumoto by the help in including his patients from the private clinic. Finally, we are grateful to Dr. Roger Emery by his support during all translation process.

References

• ¹
  Luime JJ, Koes BW, Hendriksen IJ, Burdorf A, Verhagen AP, Miedema HS, et al. Prevalence and incidence of shoulder pain in the general population; a systematic review. Scand J Rheumatol. 2004;33(2):73-81.
• ²
  Miranda H, Viikari- Juntura E, Heistaro S, Heliövaara M, Riihimäki H. A population study on differences in the determinants of a specific shoulder disorder versus nonspecific shoulder pain without clinical findings. Am J Epidemiol. 2005;161(9):847-55.
• ³
  Cools AM, Struyf F, De Mey K, Maenhout A, Castelein B, Cagnie B. Rehabilitation of scapular dyskinesis: from the office worker to the elite overhead athlete. Br J Sports Med. 2014;48(8):692-7.
• ⁴
  Kirkley A, Griffin S, Dainty K. Scoring systems for the functional assessment of the shoulder. Arthroscopy. 2003;19(10):1109-20.
• ⁵
  Terwee CB, Bot SD, de Boer MR, van der Windt DA, Knol DL, Dekker J, et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. 2007;60(1):34-42.
• ⁶
  Slobogean GP, Slobogean BL. Measuring shoulder injury function: common scales and checklists. Injury. 2011;42(3):248-52.
• ⁷
  Fayad F, Macé Y, Lefevre-Colau MM. Shoulder disability questionnaires: a systematic review. Ann Readapt Med Phys. 2005;48(6):298-306.
• ⁸
  Constant CR, Murley A. Constant score for shoulder evaluation. Clin Orthop Relat Res. 1987;(214):160-4.
• ⁹
  Constant CR, Gerber C, Emery RJ, Sojbjerg JO, Gohlke F, Boileau P. A review of the Constant score: modifications and guidelines for its use. J Shoulder Elbow Surg. 2008;17(2):355-61.
• ¹⁰
  Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross- cultural adaptation of self- report measures. Spine (Phila Pa 1976). 2000;25(24): 3186-91.
• ¹¹
  Guillemin F, Bombardier C, Beaton D. Cross- cultural adaptation of health- related quality of life measures: literature review and proposed guidelines. J Clin Epidemiol. 1993;46(12):1417-32.
• ¹²
  Cook DA, Beckman TJ. Current concepts in validity and reliability for psychometric instruments: theory and application. Am J Med. 2006;119(2):166.e7-16.
• ¹³
  Barten JA, Pisters MF, Huisman PA, Takken T, Veenhof C. Measurement properties of patient-specific instruments measuring physical function. J Clin Epidemiol. 2012;65(6):590-601.
• ¹⁴
  Placzek JD, Lukens SC, Badalanmenti S, Roubal PJ, Freeman DC, Walleman KM, et al. Shoulder outcome measures: a comparison of 6 functional tests. Am J Sports Med. 2004;32(5):1270-7.
• ¹⁵
  Floyd F. Factor analysis in the development and refinement of clinical assessment instruments. Psychol Assessment. 1995;7(3):286-99.
• ¹⁶
  Moeller AD, Thorsen RR, Torabi TP, Bjoerkman AS, Christensen EH, Maribo T, et al. The Danish version of the modified Constant-Murley shoulder score: reliability, agreement, and construct validity. J Orthop Sports Phys Ther. 2014;44(5):336-40.
• ¹⁷
  Ban I, Troelsen A, Christiansen DH, Svendsen SW, Kristensen MT. Standardised test protocol (Constant Score) for evaluation of functionality in patients with shoulder disorders. Dan Med J. 2013;60(4):A4608.
• ¹⁸
  Mokkink LB, Terwee CB, Knol DL, Stratford PW, Alonso J, Patrick DL, et al. The COSMIN checklist for evaluating the methodological quality of studies on measurement properties: a clarification of its content. BMC Med Res Methodol. 2010;10:22.
• ¹⁹
  Bryman A, Cramer D. Quantitative data analysis with SPSS 12 and 13: a guide for social scientists. New York: Routledge; 2005.
• ²⁰
  Cronbach L. Coefficient alpha and the internal structure of tests. Psychometrika. 1951;16(3):297-334.
• ²¹
  Cronbach L. Internal consistency of tests: analyses old and new. Psychometrika. 1988;53(1):63-70.
• ²²
  Orfale AG, Araujo PM, Ferraz MB, Natour J. Translation into Brazilian Portuguese, cultural adaptation and evaluation of the reliability of the Disabilities of the Arm, Shoulder and Hand Questionnaire. Braz J Med Biol Res. 2005;38(2): 293-302.
• ²³
  Norris M, Lecavalier L. Evaluating the use of exploratory factor analysis in developmental disability psychological research. J Autism Dev Disord. 2010;40:8-20.
• ²⁴
  Hair JF. Multivariate data analysis. 7 th ed. Upper Saddle River, NJ: Prentice Hall; 2010.
• ²⁵
  Livain T, Pichon H, Vermeulen J, Vaillant J, Saragaglia D, Poisson MF, et al. Intra- and interobserver reproducibility of the French version of the Constant- Murley shoulder assessment during rehabilitation after rotator cuff surgery. Rev Chir Orthop Reparatrice Appar Mot. 2007;93(2):142-9.
• ²⁶
  Razmjou H, Bean A, Macdermid JC, van Osnabrugge V, Travers N, Holtby R. Convergent validity of the Constant- Murley outcome measure in patients with rotator cuff disease. Physiother Can. 2008;60(1):72-9.
• ²⁷
  Roy JS, MacDermid JC, Woodhouse LJ. A systematic review of the psychometric properties of the Constant- Murley score. J Shoulder Elbow Surg. 2010;19(1):157-64.
• ²⁸
  Dawson J, Fitzpatrick R, Carr A. Questionnaire on the perceptions of patients about shoulder surgery. J Bone Joint Surg Br. 1996;78(4):593-600.
• ²⁹
  Christie A, Hagen KB, Mowinckel P, Dagfinrud H. Methodological properties of six shoulder disability measures in patients with rheumatic diseases referred for shoulder surgery. J Shoulder Elbow Surg. 2009;18(1):89-95.
• ³⁰
  Othman A, Taylor G. Is the constant score reliable in assessing patients with frozen shoulder? 60 shoulders scored 3 years after manipulation under anaesthesia. Acta Orthop Scand. 2004;75(1):114-6.
• ³¹
  Rocourt MH, Radlinger L, Kalberer F, Sanavi S, Schmid NS, Leunig M, et al. Evaluation of intratester and intertester reliability of the Constant- Murley shoulder assessment. J Shoulder Elbow Surg. 2008;17(2):364-9.
• ³²
  Holmgren T, Oberg B, Adolfsson L, Bjornsson Hallgren H, Johansson K. Minimal important changes in the Constant- Murley score in patients with subacromial pain. J Shoulder Elbow Surg. 2014;23(8):1083-90.
• ³³
  Puga VO, Lopes AD, Costa LO. Assessment of cross- cultural adaptations and measurement properties of self- report outcome measures relevant to shoulder disability in Portuguese: a systematic review. Rev Bras Fisioter. 2012;16(2):85-93.
• ³⁴
  de Souza MB, Martins J, Hotta GH, de Oliveira AS. Measurement properties of the Brazilian Version of the Penn Shoulder Score (PSS-Brazil): reliability, validity, and responsiveness. J Orthop Sports Phys Ther. 2015;45(2):137-42.
• ³⁵
  Moser AD, Knaut LA, Zotz TG, Scharan KO. Validity and reliability of the Portuguese version of the American Shoulder and Elbow Surgeons Standardized Shoulder Assessment Form. Rev Bras Reumatol. 2012;52(3):348-56.
• ³⁶
  Neto JO, Gesser RL, Steglich V, Bonilauri Ferreira AP, Gandhi M, Vissoci JR, et al. Validation of the Simple Shoulder Test in a Portuguese-Brazilian population. Is the latent variable structure and validation of the Simple Shoulder Test Stable across cultures? PLoS ONE. 2013;8(5):e62890.

• ☆
  This study was performed at Universidade Federal de Ciências da Saúde de Porto Alegre (UFSCPA), Porto Alegre, RS, Brazil.

Appendix A. Supplementary data

Supplementary material associated with this article can be found in the online version available at doi: doi:10.1016/j.rbo.2015.11.004.

Publication Dates

• Publication in this collection
  Sep-Oct 2016

History

• Received
  26 Sept 2015
• Accepted
  10 Nov 2015