Abstract
Aim: The purpose of this paper was to analyze the internal consistency of the Sport Motivation Scale-II (SMS-II) questionnaire based on three studies with sports practitioners in the Brazilian college context.
Methods: All the three studies i) were carried out with students enrolled in higher education and engaged in sports training by their institution (n1 = 304; n2 = 441; n3 = 310); ii) used SMS-II as an instrument for data collection; iii) used the online procedure for data collection. The Confirmatory Factor Analysis was performed and Cronbach's Alpha (α), McDonald's Omega (Ω), and the correlation between items were used to assess internal consistency.
Results: As potentialities, five dimensions of the SMS-II presented high values of internal reliability (α and Ω > 0.60). As a limitation, the dimension of introjected regulation showed low reliability (α and Ω < 0.40) in the three studies conducted by our group. The exclusion of item 16 enabled an increase in alpha, but without achieving satisfactory reliability in Study 1 (α = 0.46) and Study 2 (α = 0.31). In Study 3, the exclusion of item 7 improved the internal reliability of the dimension (α = 0.43) also without reaching satisfactory values of internal consistency. It was not possible to identify a problematic item, given that each item had a different influence in the three studies.
Conclusion: We found good values of validity and internal consistency for the Brazilian version of the SMS-II. However, we suggest attention to the introjected regulation dimension when using the instrument in the Brazilian college context.
Keywords reliability; college; motivation; sport psychology
Introduction
There is a growing body of literature investigating the influence of motivation on the development, maintenance, or drop out of sports practice1. Scholars have investigated the differences between athletes that focus on improving their performance, while others only try to avoid failing; or why some may persist when facing a failure, while others give up or lose interest1. To investigate this complex relationship between motivation and sports practice, researchers resort to the Theory of Self-Determination (TSD) allowing to discriminate the different reasons for engagement and interest in sport through a motivation continuum. According to this theory, on the one hand, athletes can be motivated by external factors, such as rewards, assessments, pressure from parents or coaches, or opinions about their ability and competence in sport2. On the other hand, they can also be driven by interest, curiosity, and a desire for self-control and improvement - known as intrinsic factors2. Based on the TSD, it is possible to understand the intrinsic and extrinsic factors for participation in sport, considering intrinsic factors the motivation to do something because it is inherently interesting or enjoyable, and extrinsic factors the motivation to do something as a means to an end or a reward2.
The applicability of the theory in the sports context has generated the need to build a scale that measures the different motivations and provides reliable information for psychologists and professionals involved in sports training. The original version of the Sport Motivation Scale (SMS) was created in French and English simultaneously3,4. The French version was validated in a group of more than 500 college athletes representing a variety of sports3. The English version was validated through two studies with more than 600 English-speaking athletes4. The factor analysis performed on both studies led to the proposal of seven factors, each one with four items, totaling 28 items. The seven factors measured the three types of intrinsic motivation (to know, to experience stimulation, and to perform), three types of extrinsic regulation (external regulation, introjected regulation, and identified regulation), and amotivation.
Despite the rigorous process of translation and validation to create the scale, several researchers questioned the psychometric properties of the SMS. Mallett et al.5 indicated that the scale needed to be revisited to include an integrated regulation measure because, without this measure, the instrument did not represent all the constructs of TSD. In addition, the authors suggested the exclusion of some items and proposed a revised version, entitled SMS-65. These limitations led the authors, of the original version of the scale, to revise the instrument and, based on the new analysis, to propose a second version called SMS-II6. In this new version, the instrument consists of 18 items that comprise six factors named amotivated, external, introjected, identified, integrated, and intrinsic regulation. The SMS-II presented a better adjustment of the model than the original version in all indicators and greater reliability between the subscales. The results also demonstrated that the new measure of intrinsic regulation, as well as the measure of integrated regulation, measured distinct and unique concepts compared to the other dimensions of the instrument.
Since then, the instrument has been translated and validated for different countries, such as France1, Iran7, Spain8, Turkey9, Mexico10, and Hungary11. In the Brazilian context, SMS-II was translated and validated by Nascimento Junior et al.12. The authors presented the validity and reliability of the adapted instrument, but they showed that Cronbach's Alpha of the external regulation (0.69) and introjected regulation (0.61) dimensions were close to the ones recommended by the literature (0.70), suggesting inconsistencies in the instrument format for the Brazilian context. The authors also highlighted issues regarding the introjected dimension when performing the Confirmatory Factor Analysis (CFA). They found factorial loads above 0.5 for items 1 and 7 (introjected dimension) and lower values of correlation between the measurement errors of the items, suggesting the need to review this dimension, either in the translation or in the understanding of the concept by the Brazilian athletes.
We found few national studies that used SMS-II to assess the motivation of Brazilian sports practitioners and when they did, it was not reported the measures of internal consistency of the instrument13,14. Others Brazilian studies reported general Cronbach's Alpha of the instrument and did not show the reliability for each scale15,16. Further studies should investigate the psychometric properties of the SMS-II version translated into Brazilian Portuguese, ensuring that its dimensions are consistent and provide reliable information for sports psychologists. Therefore, this paper aimed to analyze the internal consistency of the SMS-II questionnaire based on three studies carried out with sports practitioners in the Brazilian college context.
Methods
To assess the internal consistency of the SMS-II questionnaire, we conducted three studies with college sports practitioners. The studies had different general objectives, were conducted with different samples, and, consequently, were approved by different Ethics and Research Committees. However, all studies i) were carried out with students enrolled in a higher education institution and engaged in sports training by their institution, ii) used SMS-II as an instrument for data collection; iii) used the online procedure for data collection; iv) accessed the participants using the convenience and snowball sampling.
Study 1
Participants were 304 college students aged between 18 and 37 years (126 women, 178 men; M = 21.8 and SD = 2.3 years old). All participants studied in private universities. This study was approved by the institutional research and ethics committee (CAAE: 37305320.6.0000.0084).
Study 2
Participants were 441 college students aged between 18 and 35 years (263 women, 178 men; M = 21.8 and SD = 3.9 years old), 293 (65%) study in public institution and 148 (35%) in private institutions. This study was approved by the institutional research and ethics committee (CAAE: 29832620.1.0000.5404).
Study 3
Participants were 310 college students aged between 18 and 30 years (140 women, 170 men; M = 21.9 and SD = 2.7 years old), 250 (81%) study in public institution and 60 (19%) in private institutions. This study was approved by the institutional research and ethics committee (CAAE: 09247219.7.0000.5404)
Procedures
The questionnaire and the Informed Consent Form (ICF) were adapted to the Google Forms platform. First, the participant accessed the full and online version of the ICF through Google Forms. After a careful reading and the acceptance of the ICF, they were directed to the questionnaire on the same platform. We used the instrument adapted to Portuguese by Nascimento Junior et al.12 for data collection. The questionnaire starts with the introductory question “Why do you play sports?”. The instrument comprises 18 items on the 7-point Likert scale, which vary between 1 (strongly disagree) and 7 (totally agree) and are grouped into 6 dimensions (3 items for each construct) corresponding to the intrinsic, integrated, identified, introjected and extrinsic regulations, and amotivated.
Statistical analysis
The normality of the data was analyzed by the Kolmogorov-Sminorv test. Descriptive statistics (mean and standard deviation) were used to present the data. The Confirmatory Factor Analysis was used to analyze the factorial structure of the six dimensions based on the original model proposed by Pelletier et al.6 and validated to Portuguese by Nascimento Junior et al.12. We run two models based on the maximum likelihood estimation12. The models were evaluated using different fit indices, such as Chi-Square (χ2) and Chi-Square/Degrees of Freedom (df) (χ2/DF) given the influence of the sample size on χ217. We considered good fit values for χ2/DF those above 5.018. The Goodness-of-Fit index (GFI) and Adjusted Goodness-of-Fit index (AGFI) were also evaluated considering acceptable values 0.9. The Root Mean Square Error of Approximation (RMSEA) was calculated and considered acceptable with values equal to or lower than 0.0719. Tucker-Lewis's index (TLI) and Confirmatory Fit Index (CFI) were calculated and values higher than 0.90 were considered good fit values19. Akaike information criterion (AIC) and Bayesian Information Criterion (BIC) were used as estimators of prediction error. At last, the Model Expected Cross Validation Index (MECVI) was also calculated.
The internal consistency of the SMS-II dimensions was analyzed using the Cronbach's Alpha (α) and given the possible influence of the small number of items, we calculated the McDonald's Omega (Ω). To interpret both indices we used the value of 0.7 as the lower limit of internal consistency17. After finding unsatisfactory reliability in the introjected regulation, we calculated the Cronbach's Alpha of this dimension when each item was excluded to test the influence of each item on the internal consistency. Finally, we used the item-dimension correlation to analyze the association between the three items of the introjected regulation dimension and with the other items of the instrument. The significance level of 0.05 was adopted. All analyzes were performed using the MATLAB program.
Results
The results of the CFA are shown in Table 1. The initial model (1) presented marginal values in most of the indices. After a visual inspection in the modification indices and tracking covariances adjustments, the second model (2) showed a better fit for the three studies reaching marginal to satisfactory values.
Table 2 presents the descriptive statistics and internal consistency of the six dimensions of the SMS II questionnaire in the three studies. We found α and Ω greater than 0.60 in all dimensions in the three studies, except for the introjected regulation. The introjected regulation showed a low internal consistency with α and Ω below 0.40 in the three studies.
Internal consistency of the six dimensions of the SMS II questionnaire in the three studies.
Table 3 shows the α of the introjected regulation dimension when each item is excluded from the analysis. We found that the items have different influences in each study. The exclusion of item 16 led to an increase in α in Studies 1 and 2, but without achieving satisfactory reliability. In Study 3, the exclusion of item 7 improved the internal reliability of the dimension, but also without achieving satisfactory reliability.
Table 4 shows the correlation between the items of the introjected regulation dimension. When analyzing the item-dimension correlation (1, 7, and 16) of the introjected regulation, we found coefficients below 0.30 in the three studies. The correlation analysis in relation to the other items reveals a strong correlation between items 16 and 17 in the three studies.
Discussion
Based on these three studies conducted by our group, we found well-fitted CFA models and a good internal consistency for five dimensions of SMS-II. However, we found that the introjected regulation dimension of the SMS-II questionnaire presents a low internal consistency when used with athletes in the Brazilian college context. We found a low association between all items in the dimension, not being possible to identify an item responsible for its low consistency.
The CFA showed adequate fit indices corroborating the good validity of SMS-II found in the original version2 and the Brazilian translation and validation12. The CFA is an important analysis that supports latent models with pre-established features, evaluating the latent functioning and allowing to consider explicit errors in the format of the measure, being one of the most recommended methods to analyze the psychometric properties of SMS-II2,12. The results of our studies reinforce the reliability of the SMS-II Brazilian version proposed. However, the previous inconsistencies concerning the psychometric properties of the transcultural adaptation to the Portuguese language in the dimension of introjected regulation still persist.
The introjected regulation showed a low internal consistency with Cronbach's Alpha and McDonald's Omega below 0.40 in the three studies presented in this paper. These values were below 0.70, the lower limit indicated by Hair et al.17 as a good internal consistency of the instrument. In the three studies, we used the Portuguese version translated and validated by Nascimento Junior et al.12 and corroborate their previous finding suggesting inconsistencies in the instrument's format for the Brazilian context. However, when analyzing translation and validation studies for other cultural contexts, similar problems were found in this dimension.
In the translation and validation of the questionnaire for Chilean college students, the authors found a Cronbach's Alpha of 0.39 for the introjected dimension20. In the Spanish translation and validation, the dimensions of external regulation (0.53) and introjected (0.64) also showed the lowest internal consistencies8. The authors8 tested different adjustment models for the SMS-II and the model with 5 factors (excluding Introjected regulation and Item 15 of the scale) achieved a better model fit (GFI = 0.971; RMSEA = 0.44; AGFI = 0.955; TLI = 0.954; CFI = 0.966; CMIN/DF = 2.498; AIC = 2244.378; explained variance of 66%) compared to the 6-factor model (GFI = 0.948; RMSEA = 0.54; AGFI = 0.926; TLI = 0.91; CFI = 0.93; CMIN/DF = 3.272; AIC = 494.657; explained variance of 63%).
The authors of the original version of SMS-II observed high Cronbach's Alpha coefficients for each of the subscales except for the introjected dimension, which presented a lower value but was still at an acceptable level6. This indicates that the low Alpha found in questionnaires from different nationalities possibly reflect issues in the translation and adaptation process, and not necessarily in the construct itself. A recent study of translation and validation for Portugal showed good validity and reliability in all dimensions21, suggesting that some differences in the translation into Portuguese may assure the reliability of the introjected dimension.
Our study and previous ones used Cronbach's Alpha to assess the internal consistency of instruments, being one of the most used measures to analyze the reliability of measurement instruments. However, it is important to recognize its limitation, being influenced by the number of items evaluated17. Due to the low number of items per dimension, we also calculated McDonald's Omega and found similar results corroborating the low internal consistency of the introjected dimension.
The analysis of the item-dimension correlation of the introjected regulation showed coefficients below 0.30 corroborating the low internal consistency of the construct. According to Hair et al.17, item-dimension correlation coefficients above 0.50 are considered indicators of a well-defined structure, considering that the factor explains at least 25% of the item's variance. When analyzing Cronbach's Alpha after excluding each item, we did not find a pattern among the three studies. In two studies, the exclusion of item 16 enabled an increase in Alpha, while the exclusion of item 17 improved internal consistency in the third study. A similar scenario is found in international literature. In the translation and validation of the questionnaire into Spanish from Mexico, the authors eliminated item 16 of the introjected regulation dimension since it did not meet the criteria for a correlation between item10. According to the authors, the introjected regulation dimension should be analyzed and, possibly, reviewed, since its reliability is questionable. In the translation and validation for Turkey, the introjected regulation also had the lowest Alpha (0.44) and still had item 7 eliminated due to its low correlation between items9. These findings indicate that there is no specific item responsible for the low internal consistency of the dimension, but that different items can contribute to this problem according to the investigated context.
Although our study produced interesting findings of the SMS-II questionnaire in the Brazilian context, it is important to recognize the limitations of the present investigation. We used an online procedure for data collection, and we do not know the circumstances and contexts that the questionnaire was answered. We suggest that future studies protocols based on a face-to-face collection to compare the current findings.
Conclusions
In this study, we investigated the reliability of the SMS-II questionnaire. As a potential of the instrument, five of the six dimensions have good values of validity and internal consistency. As a limitation, the dimension of introjected regulation showed low reliability in the three studies conducted by our group. It was not possible to identify a problematic item, given that each item had a different influence in the three studies. We suggest attention to this dimension when using it in the Brazilian college context.
References
- 1. Pelletier LG, Meredith R, Camille G, Catherine H, Philippe S. French adaptation and validation of the Sport Motivation Scale-II (Echelle de Motivation Dans Les Sports-II). Int J Sport Exe Psychol. 2017;17(3):232-49.
- 2. Ryan RM, Deci EL. Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. Am Psychol. 2000;55(1):68.
- 3. Briere NM, Vallerand RJ, Blais MR, Pelletier LG. Developing and validating a measure of intrinsic and extrinsic motivation in sports: a Scale of Motivation for Sports (EMS). Int J Sport Psychol. 1995;26(4):465-89.
- 4. Pelletier LG, Kim MT, Michelle SF, Robert JV, Nathelie MB, Marc RB. Toward a new measure of intrinsic motivation, extrinsic motivation, and amotivation in sports: the Sport Motivation Scale (SMS). J Sport Exe Psychol. 1995;17(1):35-53.
- 5. Mallett C, Kawabata M, Newcombe P, Otero-Forero A, Jackson S. Sport Motivation Scale-6 (SMS-6): a revised six-factor sport motivation scale. Psychol Sport Exer. 2007;8(5):600-14.
- 6. Pelletier LG, Meredith AR, Robert JV, Edward LD, Richard MR. Validation of the revised Sport Motivation Scale (SMS-II). Psychol Sport Exer. 2013;14(3):329-41.
- 7. Manouchehri J, Farshad T, Sahar S. Validity and reliability of measurement instrument for Sport Motivation Scale in professional athletes in team sports of Iran. Afri J Psychol. 2015;18(2):2-5.
- 8. Viciana J, Mayorga-Vega D, Guijarro-Romero S, Martínez-Baena A, Blanco H. The Spanish adaptation of the Sport Motivation Scale-II in adolescent athletes. Psychol Rep. 2017;120(5):943-65.
- 9. Yıldız A, Altıntas; A, Elmas S, Asçı FH. Investigating The psychometric properties of Sport Motivation Scale-II. Turk J Sports Med. 2019;54(1):33-44.
- 10. Pineda-Espejel HA, Alarcón E, López-Ruiz Z, Trejo M, Chávez C. Propiedades psicométricas de la Escala de Motivación en el Deporte revisada (SMS-II) adaptada al Español hablado en México. RICYDE: Rev Int Cien Deporte. 2016;12(44):107-20.
-
11. Paic R, Kajos A, Meszler B, Prisztóka G. Validation of the Hungarian Sport Motivation Scale (H-SMS). Cognition Brain Behavior. 2017;21(4):275-91. doi
» https://doi.org/10.24193/cbb.2017.21.17 - 12. Nascimento Junior JRAD, Vissoci JRN, Balbim GM, Moreira CR, Pelletier L, Vieira, LF. Adaptação transcultural e análise das propriedades psicométricas da Sport Motivation Scale-II no contexto brasileiro. Rev Edu FísicaM. 2014;25(3):441-58.
-
13. Barbosa VBC, Contessoto LC, Anversa ALB, Refundini, LB. Diferença motivacional entre praticantes iniciantes e veteranos do Taekwondo. Cadernos Educ Física Esporte. 2018;16(1):41-6. doi
» https://doi.org/10.36453/2318-5104.2018.v16.n1.p41 - 14. Campos CFR, Chiminazzo JGC, Conte M, Fernandes PT. Motivação em pilotos de motovelocidade. Rev Bras Psicol Esporte. 2020;10(3):296-307.
- 15. Nascimento Junior JRAD, Silva EC, Freire GLM, Granja CTL, Silva AA, Oliveira DV. Athlete's motivation and the quality of his relationship with the coach. Apunts. 2020;142(4):21-8.
- 16. Nascimento Junior JRAD, Freire GLM, Granja CTL, Oliveira DV, Costa LGT. The role of resilience on motivation among brazilian athletics and swimming parathletes. Rev Edu Física/UEM. 2021;32(1):e3201.
- 17. Hair JF, William CB, Barry JB, Rolph EA, Ronald LT. Multivariate data analysis. 6th ed. New York, Prentice-Hall; 2009.
- 18. Tabachnick BG, Fidell LS, Ullman JB. Using multivariate statistics. 5th ed. Boston, Pearson; 2007.
- 19. Hu LT, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct. Equ. Model. 1991;6(1):1-55.
- 20. Vallejo-Reyes FA, Campbell JIM, Lochbaum MR, Duclos-Bastías DM, Guerrero-Santana I, Beltrán HJC. Adaptación y validación de la Escala de Motivación en el Deporte 2 (EMD-2) para estudiantes universitarios chilenos. Cua Psychol Deporte. 2018;18(1):63-74.
- 21. Rodrigues F, Pelletier L, Rocchi M, Cid L, Teixeira D, Monteiro D. Adaptation and validation of a Portuguese version of the Sports Motivation Scale-II (SMS-II-P) showing invariance for gender and sport type. Percept Motor Skills. 2021;128:2669-87
-
Associate Editor: Ricardo Augusto Barbieri0000-0003-3552-7585, Estácio UniSEB, Ribeirão Preto, SP, Brazil. E-mail: barbieri_ef@hotmail.com.
Publication Dates
-
Publication in this collection
17 Jan 2022 -
Date of issue
2022
History
-
Received
02 July 2021 -
Accepted
17 Nov 2021