Acessibilidade / Reportar erro

Evidence of Validity of the Knowledge Assessment Instrument on Metabolic Syndrome

Evidências de Validade do Instrumento de Avaliação de Conhecimentos sobre Síndrome Metabólica

Evidencia de validez del Instrumento de Evaluación de Conocimientos sobre Síndrome Metabólico

Abstract:

This study aimed to develop and validate an instrument to analyze adults’ knowledge about metabolic syndrome. The instrument was developed via documentary research, resulting in 40 statements with Likert-type responses, organized into three domains: (1) knowledge about obesity and related diseases; (2) knowledge about risk factors; and (3) knowledge about protective factors. Experts’ analysis of the construct and testing on 600 participants were used to validate the instrument. Exploratory factor analysis indicated that six assertions had very low factor loadings and were excluded, resulting in 34 assertions. The final version of the instrument presented satisfactory internal consistency indicators, guaranteeing the first evidence of the instrument accuracy in assessing adults’ knowledge about metabolic syndrome.

Keywords:
patient health questionnaire; metabolic syndrome; health promotion; non communicable diseases

Resumo:

O objetivo do presente estudo foi desenvolver e validar um instrumento para analisar o conhecimento de indivíduos adultos sobre a síndrome metabólica. O instrumento foi desenvolvido por meio de uma pesquisa documental resultando na construção de 40 assertivas com respostas tipo Likert, organizadas em três domínios: (1) conhecimento sobre obesidade e doenças relacionadas; (2) conhecimento sobre fatores de risco; e (3) conhecimento sobre fatores de proteção. A validação foi realizada por meio da análise do constructo por especialistas e teste em uma amostra de 600 adultos. A análise fatorial exploratória indicou que seis assertivas apresentaram cargas fatoriais muito baixas e foram excluídas resultando em 34 assertivas. A versão final do instrumento apresentou indicadores de consistência interna satisfatórios, garantindo as primeiras evidências de precisão do instrumento para se avaliar o conhecimento de indivíduos adultos sobre a síndrome metabólica.

Palavras-chave:
questionário de saúde do paciente; síndrome metabólica; promoção da saúde; doenças crônicas não transmissíveis

Resumen:

El objetivo de este estudio fue desarrollar y validar un instrumento para identificar el conocimiento de los adultos sobre el síndrome metabólico. El instrumento se desarrolló en una investigación documental que resultó en la construcción de cuarenta afirmaciones con respuestas tipo Likert, organizadas en tres dominios: (1) conocimiento sobre la obesidad y enfermedades relacionadas; (2) conocimiento sobre factores de riesgo; y (3) conocimiento sobre factores protectores. La validación fue realizada por expertos mediante análisis del constructo y testeo sobre una muestra de 600 adultos. El análisis factorial exploratorio indicó que seis afirmaciones tenían cargas factoriales muy bajas y se excluyeron, lo que resultó en 34 afirmaciones. La versión final del instrumento presentó indicadores de consistencia interna satisfactorios, lo que garantiza la primera evidencia de la precisión del instrumento para evaluar el conocimiento de los adultos sobre el síndrome metabólico.

Palabras clave:
cuestionario de salud del paciente; síndrome metabólico; promoción de la salud; enfermedades no transmisibles

Metabolic syndrome (MS) is a clinical condition characterized by a complex set of metabolic disorders associated with the development of several comorbidities, especially diabetes and cardiovascular diseases (Mahadzir et al., 2020 Mahadzir, M. D. A., Quek, K. F., & Ramadas, A. (2020). Nutrition and lifestyle behavior peer support program for adults with Metabolic Syndrome: Outcomes and lessons learned from a feasibility trial. Nutrients, 12(4), 1091. https://doi.org/10.3390/nu12041091
https://doi.org/10.3390/nu12041091...
). Recently, the prevalence of MS has increased significantly in the world population and the worsening of its components has contributed to increased mortality. Therefore, the burden resulting from the consequences associated with MS overloads the health system, leading to significant expenses (Nilson et al., 2020 Nilson, E. A. F., Andrade, R. C. S., Brito, D. A., & Oliveira, M. L. (2020). Custos atribuíveis a obesidade, hipertensão e diabetes no Sistema Único de Saúde, Brasil, 2018. [Costs attributable to obesity, hypertension and diabetes in the Unified Health System, Brazil, 2018] Revista Panamericana de Salud Pública, 44, e32. https://doi.org/10.26633/RPSP.2020.32
https://doi.org/10.26633/RPSP.2020.32...
), which makes MS an important global public health issue.

With a complex etiology, MS seems to occur mainly in response to the combination of genetic and/or epigenetic factors and lifestyle (Ambrosini et al., 2020 Ambrosini, S., Mohammed, S. A., Lüscher, T. F., Costantino, S., & Paneni, F. (2020). New mechanisms of vascular dysfunction in cardiometabolic patients: Focus on epigenetics. High Blood Pressure & Cardiovascular Prevention, 27(5), 363–371. https://doi.org/10.1007/s40292-020-00400-2
https://doi.org/10.1007/s40292-020-00400...
), such as an unbalanced diet — high intake of processed foods, ultra-processed foods, saturated and trans fats (Semnani-Azad et al., 2020 Semnani-Azad, Z., Khan, T. A., Blanco Mejia, S., Souza, R. J., Leiter, L. A., Kendall, C. W. C., Hanley, A. J., & Sievenpiper, J. L. (2020). Association of major food sources of fructose-containing sugars with incident metabolic syndrome. JAMA Network Open, 3(7), e209993. https://doi.org/10.1001/jamanetworkopen.2020.9993
https://doi.org/10.1001/jamanetworkopen....
) — and sedentary behaviors (Amirfaiz & Shahril, 2019 Amirfaiz, S., & Shahril, M. R. (2019). Objectively measured physical activity, sedentary behavior, & metabolic syndrome in adults: Systematic review of observational evidence. Metabolic Syndrome and Related Disorders, 17(1), 1–21. https://doi.org/10.1089/met.2018.0032
https://doi.org/10.1089/met.2018.0032...
). In this sense, many of the MS control interventions are based on lifestyle changes, and theirs success depends on the degree of people’s self-care concerning their own health. Self-care can be defined as the process of maintaining health by conscious decision-making for health management (Jaarsma et al., 2020 Jaarsma, T., Westland, H., Vellone, E., Freedland, K. E., Schröder, C., Trappenburg, J. C. A., Stromberg, A., & Riegel, B. (2020). Status of theory use in self-care research. International Journal of Environmental Research and Public Health, 17(24), 9480. https://doi.org/10.3390/ijerph17249480
https://doi.org/10.3390/ijerph17249480...
), which, in the case of MS, can be exemplified by adherence to health-promoting practices.

Admittedly, strategies based on health promotion, according to sociocultural determinants of the health-disease process, contribute to the control of many health issues, as they expand the individual’s ability to analyze reality and make more conscious decisions (Vieira et al., 2021 Vieira, M. S. N.,Matias, K. K., & Queiroz, M. G. (2021). Educação em saúde na rede municipal de saúde: Práticas de nutricionistas. [Health education in the municipal health network: Practices of nutritionists.] Ciência & Saúde Coletiva, 26(2), 455–464. https://doi.org/10.1590/1413-81232021262.41062020
https://doi.org/10.1590/1413-81232021262...
). Within a comprehensive sociopolitical process — with actions based on changing social, environmental, and economic determinants of health — health promotion has supported the optimization of healthcare services to improve quality of life (Nutbeam&Muscat, 2021 Nutbeam, D., & Muscat, D. M. (2021). Health Promotion Glossary 2021. Health Promotion International, 36(6), 1457-1598. https://doi.org/10.1093/heapro/daaa157
https://doi.org/10.1093/heapro/daaa157...
). Therefore, health promotion, by encouraging healthier lifestyle habits, can lead to significant changes in people’s daily lives and, in the long term, could even be decisive in the MS management.

Analyzing the efficiency of the health promotion approach in the MS prevention and/or treatment is a relevant step to enable the implementation of increasingly effective strategies to control this important public health issue, requiring the development of accurate and easy-to-apply assessment instruments to determine people’s knowledge about MS. Regarding aspects of health promotion linked to the individuals’ and communities’ training, there is a need for valuing knowledge related to the health-disease process, with a primary focus on risk and protective factors, since the literature has pointed out that the level of knowledge on MS differs worldwide, both in young people and older people (Anand et al., 2023 Anand, V. V., Goh, R. S. J., Nah, B., Koh, S. W. C., Lim, J., Neo, N. W. S., Chew, J., Lee, Y. Y., Chin, Y. H., Chong, B., Kong, G., Tan, B., Low, Z., Khoo, C. M., Goh, L. H., Loh, P. H., Chai, P., Dalakoti, M., Chan, M., … Chew, N. W. S. (2023). General public’s knowledge, awareness, and perception of cardiometabolic diseases: Data from a Singapore study population. Frontiers in Medicine, 10, 1193829. https://doi.org/10.3389/fmed.2023.1193829
https://doi.org/10.3389/fmed.2023.119382...
; Cortez et al., 2018 Cortez, R. M. A., Batista, A. M. O., Rocha, M. R., Santos, I. M. S., Moura, J. R. A., & Silva, A. R. V. (2018). Students’ knowledge of metabolic syndrome after educational intervention. Revista Brasileira de Enfermagem, 71(Suppl 4), 1493-1499. http://dx.doi.org/10.1590/0034-7167-2017-0342
http://dx.doi.org/10.1590/0034-7167-2017...
; Yahia et al., 2014 Yahia, N., Brown, C., Rapley, M., & Chung, M. (2014). Assessment of college students’ awareness and knowledge about conditions relevant to metabolic syndrome. Diabetology & Metabolic Syndrome, 6, 111. https://doi.org/10.1186/1758-5996-6-111
https://doi.org/10.1186/1758-5996-6-111...
; Wang et al., 2019 Wang, Q., Chair, S. Y., Wong, E. M., Taylor-Piliae, R. E., Qiu, X. C. H., & Li, X. M. (2019). Metabolic Syndrome Knowledge among adults with cardiometabolic risk factors: A cross-sectional study. International Journal of Environmental Research and Public Health, 16(1), 159. https://doi.org/10.3390/ijerph16010159
https://doi.org/10.3390/ijerph16010159...
).

However, according to the authors’ knowledge, instruments that sought to assess the adults’ knowledge on MS are unknown. (Ferret et al., 2021 Ferret, J. C. F., Yamaguchi, M. U., Branco, B. H. M., & Bernuci, M. P. (2021). Interventions based on practice of resistance exercises: A systematic review. Journal of Physical Education and Sport, 21(4), 1705-1714. https://doi.org/10.7752/jpes.2021.04216
https://doi.org/10.7752/jpes.2021.04216...
). Therefore, this study aimed to develop and to validate an instrument to assess the adults’ knowledge on MS.

Method

This is a mixed study, involving documentary analysis (development of an instrument) and cross-sectional field research (instrument validation), conducted from January 2020 to February 2021 in the municipality of Maringá/PR. The study followed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) recommendations (von Elm et al., 2008 von Elm,E.,Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., Vandenbroucke, J. P., & STROBE Initiative (2008). The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: Guidelines for reporting observational studies. Journal of Clinical Epidemiology, 61(4), 344–349. https://doi.org/10.1016/j.jclinepi.2007.11.008
https://doi.org/10.1016/j.jclinepi.2007....
).

Participants

A total of 600 adults participants were recruited via social media platforms, namely: Facebook, Instagram, and WhatsApp.

Procedures

Data collection. The key elements for the development of the first version of the instrument were identified between January and March 2020, via a bibliographic search in periodicals indexed on the Scientific Electronic Library Online (SciELO), and on the the National Library of Medicine (PubMed), Medical Literature Analysis and Retrievel System Online (Medline), and Latin American and Caribbean Literature in Health Sciences (LILACS) databases. To search for articles, a combination of descriptors was used: “ síndrome metabólica ” [metabolic syndrome] and “ fatores de risco ” [risk factors], “ síndrome metabólica ” [metabolic syndrome] and “ fatores de proteção ” [protective factors], “ síndrome metabólica ” [metabolic syndrome] and “ obesidade ” [obesity], “ síndrome metabólica ” [metabolic syndrome] and “ prevenção ” [prevention], “ síndrome metabólica ” [metabolic syndrome] and “ tratamento ” [treatment]. After an in-depth reading of all scientific articles found by the search and relevant to the topic, all pieces of information about the MS prevention and treatment were identified. Those pieces of information were used as theoretical references to choose the instrument domains that were representative of what would be fundamental about the MS prevention and treatment. In this sense, 40 statements were prepared distributed into three domains: knowledge on obesity and related diseases (KORD); knowledge on risk factors (KRF), and knowledge on protective factors (KPF). Responses to the statements were made using a 5-point Likert measurement scale.

The validation process was developed during the second quarter of 2020. Initially, the theoretical analysis of the items was carried out by three experts in the treatment of obesity and related comorbidities (two physical education professionals, with master’s and doctorate degrees in the area — one of them is also a nutritionist —, and one endocrinologist, also with a master’s and doctorate degree in the area). To this end, a questionnaire was developed so that the raters could determine the clarity and degree of understanding of each statement. The instrument (1 st version) and the questionnaire were sent to raters via the Internet. They were asked to consider whether: (a) the item assessed the phenomenon measured; (b) the item was written clearly; (c) the item was written appropriately; (d) the item was relevant to the instrument, and (e) to which dimension the item should relate to KORD, KRF, or KPF. The item relevance was assessed based on a scale from 1 to 4: 1 – nonrelevant or nonrepresentative; 2 – item requires major revision to be representative; 3 – item needs a small revision to be representative; and 4 – relevant or representative. After receiving the raters’ analysis, the statements were adjusted and the second version of the instrument was developed.

The second version of the instrument was subjected to semantic and appearance validation by applying the instrument to 600 adults, recruited via Facebook, Instagram, and WhatsApp. The online version of the instrument was made available on social media from 2020 to February 2021. A sociodemographic questionnaire was also made available to determine the respondents’ characteristics.

Data analysis. To estimate the evidence of the instrument content validity, Content Validity Index (CVI) > 0.80 and Cronbach’s Alpha > 0.80 were adopted, an index considered capable of evaluating inter-rater agreement (Hernández-Nieto, 2002Hernández-Nieto, R. A. (2002). Contribuciones al análisis estadístico de datos. [Contributions to statistical data analysis.] Booksurge. ). In this study, raters evaluated the instrument regarding the clarity of language, practical suitability, and theoretical relevance of the items. Values above 0.80 were considered adequate (Cassepp-Borges et al., 2010Cassepp-Borges, V., Balbinotti, M. A. A., & Teodoro, M. L. M. (2010). Tradução e validação de conteúdo: Uma proposta para a adaptação de instrumentos. In: L. Pasquali. Instrumentação psicológica [Translation and Content Validation: A proposal for adapting instruments. In: L. Pasquali. Psychological instrumentation] (pp. 506-520). Artmed. ).

To evaluate the evidence of instrument validity, a Parallel Analysis (PA) was carried out to estimate the dimensionality of the item set (Lim & Jahng 2019 Lim, S., & Jahng, S.(2019). Determining the number of factors using parallel analysis and its recent variants. Psychological Methods, 24(4), 452–467. https://doi.org/10.1037/met0000230
https://doi.org/10.1037/met0000230...
); then, the factor structure was estimated using Exploratory Factor Analysis (EFA). After the factorial structure was found, Confirmatory Factor Analysis (CFA) was performed to verify the validity of the instrument construct by (a) factor-item parameter and individual item-item reliability; (b) absolute, incremental, and parsimonious fit indices, and (c) average variance extracted to examine convergent validity. The model fit was tested using fit indices (expected reference values for each index): Chi-squared (ꭓ² and p-value), Root Mean Square Error of Approximation (RMSEA < 0.08, 90% CI), Tucker-Lewis Index (TLI > 0.90), and Comparative Fit Index (CFI > 0.95).

Convergent validity was assessed by the Average Variance Extracted (AVE), and values greater than 0.50 were considered acceptable indicators of convergent validity (Luoma et al., 2010 Luoma, J. B., O’Hair, A. K., Kohlenberg, B. S., Hayes, S. C., & Fletcher, L. (2010). The development and psychometric properties of a new measure of perceived stigma toward substance users. Substance Use & Misuse, 45(1–2), 47–57. https://doi.org/10.3109/10826080902864712
https://doi.org/10.3109/1082608090286471...
). Discriminant validity was assessed by comparing AVE with the quadratic correlation between the factors. Composite Reliability (CR) was calculated using CFA results, as this measure provides the internal consistency index of the instrument dimensions through the factor loadings of the respective items. Values > 0.70 were considered indicators of adequate composite reliability.

Regarding the instrument precision indicators, Cronbach’s Alpha and McDonald’s Omega coefficients were calculated, with satisfactory values being ≥ 0.70 (Malacarne et al., 2017 Malacarne, M. P., Luiz, S. G., Amaral, T. R., & Siqueira, M. M. (2017). Avaliação de serviços de saúde no campo da saúde coletiva: Uma sistematização em curso [Evaluation of health services in the field of collective health: An ongoing systematization]. Revista Brasileira de Pesquisa Em Saúde, 18(1), 62–67. https://doi.org/10.21722/rbps.v18i1.15136
https://doi.org/10.21722/rbps.v18i1.1513...
). All analyses were conducted with R Language for Statistical Computing (R Foundation, Vienna).

Ethical Considerations

The study protocol was approved by the Research Ethics Committee of the Universidade Cesumar under Opinion No. 4,080,654, CAAE No. 32984620.4.0000.5539.

Results

The first results to determine the instrument validity were obtained from the CVC analysis to test inter-rater agreement. Table 1 presents the values obtained for each of the 40 statements, as well as the total CVC. Note that all items presented values ≥ 0.80, indicating that, for this analysis, acceptable values of inter-rater agreement were obtained with regard to the statement content.

After the results of evidence of content validity, and before the analyses of the instrument structure, an analysis of data adequacy was performed using the Kaiser-Meyer-Olkin index (KMO = 0.94), which indicated the suitability of the data to be subjected to AFE. Then, an analysis was carried out to verify the instrument dimensionality after applying the adapted version (2 nd version) on social media platforms via the Internet.

Table 2 presents the characterization of the 600 instrument respondents to complete the validation process. Notably, the majority of participants are aged 31 – 50 years (41%), women (71%), White (79%), and with completed Tertiary Education (48%). Regarding family income, the declared average was six minimum wages.

Based on the 600 participants’ responses, the data were processed using PA, with the results indicating the retention of two factors. Nevertheless, the structure suggested by PA did not correspond to the structure initially designed for the instrument, and, besides, the fit indices for EFA with two factors were not adequate. Thus, EFA was performed to test how the instrument behaved considering its unidimensionality, which generated satisfactory fit indices. Some statements, however, presented very low factor loadings (1, 2, 3, 5, 14, and 40), thus the option was their exclusion. Table 3 presents the values of the model estimated via AFE for each statement, with the respective factor loadings and communality indices, as well as the internal consistency indicators presented by the factor.

With the new statement structure (3rd version), the AFE results showed that the unidimensional instrument model, now with 34 statements, presented satisfactory fit indices. Regarding the values of the factor loadings of the 34 statements, all were above 0.4, and the precision indicators — Cronbach’s alpha and McDonald’s Omega — indicated satisfactory levels of internal consistency, guaranteeing the first evidence of internal structure and instrument accuracy.

Confirmatory Factor Analysis (CFA) was performed following the unidimensionality previously found. Then, adequate fit indices were found for the unidimensional scale, values presented in Table 4 .

Figure 1 shows the factor loadings of all statements performed adequately, with indices ranging from 0.44 (item 13) to 0.85 (item 11).

Table 1
Content validity coefficient of the instrument statements to test inter-rater agreement

Table 2
Characterization of study participants

Table 3
Values related to Exploratory Factor Analysis of the Instrument

Table 4
Fit indicators of confirmatory factor analysis models

Figure 1
CFA Diagram of the Instrument for Assessing Knowledge of Metabolic Syndrome

After the instrument was validated, its final version consisted of 34 statements.

Discussion

The instrument for assessing knowledge on MS, proposed in this study, was developed based on aspects associated with the etiology, treatment, and control of this chronic noncommunicable disease, which enabled both the design of the statements and the organization of the construct domains. The use of documentary analysis — especially of scientific articles — for the development of the initial versions of health assessment instruments has been suggested as an important stage in the process, not only for content enrichment but also for domain definition and organization (Leite et al., 2018 Leite, S. S., Áfio, A. C. E.; Carvalho, L. V.; Silva, J. M.; Almeida, P. C., & Pagliuca, L. M. F. (2018). Construction and validation of an educational content validation instrument in health. Revista Brasileira de Enfermagem, 71(Suppl 4), 1635–1641. https://doi.org/10.1590/0034-7167-2017-0648 .
https://doi.org/10.1590/0034-7167-2017-0...
; Rattray & Jones, 2007 Rattray, J., & Jones, M. C. (2007). Essential elements of questionnaire design and development. Journal of Clinical Nursing, 16(2), 234–243. https://doi.org/10.1111/j.1365-2702.2006.01573.x
https://doi.org/10.1111/j.1365-2702.2006...
).

Regarding “knowledge on obesity and related diseases” domain, three articles were used as main references for the elaboration of statements 2, 3, 4, 12, 13, 14, 16, 18, 19, 20, 21, 22, 23, 24, 26 and 27. These statements direct the analysis to determine relevant knowledge on MS etiology and its relationship with obesity, in addition to be related with symptoms and conditions that are syndrome-related. In the “knowledge on risk factors” domain, three articles were used as main references for the elaboration of statements 1, 10, 11, 15, 29, 31, 32, 33. This domain covers knowledge on the risk factors that can trigger MS in the short, medium, and long terms. Recognizing the level of people’s perception of risk factors is essential for designing future health education strategies based on the prevention of obesity and, consequently, MS. In the “knowledge on protective factors” domain, three articles were used to prepare statements 5, 6, 7, 8, 9, 17, 25, 28, 30, 34. In this domain, the statements lead to knowledge of the main factors described as protective against the MS development, further strengthening the analysis of events that are crucial for implementation of more assertive strategies for MS prevention and management.

The instrument was initially validated considering quantitative and qualitative questions from the perspective of content analysis based on the evaluation of suggestions from a panel of experts, as previously proposed (Rodríguez de Vera et al., 2021 Rodríguez de Vera, L., López Martínez, A. B., & Muria, A. J. (2021). Validação por expertos de uma proposta metodológica para promoção de atitudes positivas perante a deficiência em ambientes inclusivos. [Validation by experts of a methodological proposal to promote positive attitudes towards disability in inclusive environments.] Cuadernos de Psicología Del Deporte, 21(1), 162–178. https://doi.org/10.6018/cpd.436261
https://doi.org/10.6018/cpd.436261...
). This step made it possible to ensure evidence of scale validity, that is, indicate precisely what was proposed to be measured as previously indicated (Cano & Hobart, 2011 Cano, S. J., & Hobart, J. C. (2011). The problem with health measurement. Patient Preference and Adherence, 5, 279-290. https://doi.org/10.2147/PPA.S14399
https://doi.org/10.2147/PPA.S14399...
; Elangovan & Sundaravel, 2021 Elangovan, N., & Sundaravel, N. (2021). Method of preparing a document for survey instrument validation by experts. MethodsX, 8, 101326. https://doi.org/10.1016/j.mex.2021.101326
https://doi.org/10.1016/j.mex.2021.10132...
; Tibúrcio et al., 2015 Tibúrcio, M. P., Melo, G. S. M., Balduíno, L. S. C., Freitas, C. C. S., Costa, I. K. F., & Torres, G. V. (2015). Content validation of an instrument to assess the knowledge about the measurement of blood pressure. Revista de Pesquisa Cuidado é Fundamental Online, 7(2), 2475–2485. https://doi.org/10.9789/2175-5361.2015.v7i2.2475-2485
https://doi.org/10.9789/2175-5361.2015.v...
). Furthermore, to provide evidence of the instrument validity, the dimensionality of the item set was also estimated and the precision indicators were calculated (Lim & Jahng, 2019 Lim, S., & Jahng, S.(2019). Determining the number of factors using parallel analysis and its recent variants. Psychological Methods, 24(4), 452–467. https://doi.org/10.1037/met0000230
https://doi.org/10.1037/met0000230...
; Malacarne et al., 2017 Malacarne, M. P., Luiz, S. G., Amaral, T. R., & Siqueira, M. M. (2017). Avaliação de serviços de saúde no campo da saúde coletiva: Uma sistematização em curso [Evaluation of health services in the field of collective health: An ongoing systematization]. Revista Brasileira de Pesquisa Em Saúde, 18(1), 62–67. https://doi.org/10.21722/rbps.v18i1.15136
https://doi.org/10.21722/rbps.v18i1.1513...
). Taken together, all the results obtained in the process indicated that the instrument in question holds consistent validation.

According to the Standards for Educational Psychological Testing (American Educational Research Association et al., 2014American Educational Research Association, American Psychological Association, & National Council on Measurement in Education (Eds.). (2014). Standards for educational and psychological testing. American Educational Research Association. ), there are several sources of evidence that can be used when evaluating the validity of a proposed interpretation of test results for a specific use. These sources of evidence may illuminate different aspects of validity, but they do not represent distinct types of validity as this is a unitary concept. Validity is the degree to which all accumulated pieces of evidence support the intended interpretation of test results for the proposed use.

The availability of an instrument with consistent validation is important in the current scenario of increasing cases of people with obesity, since many of the efforts to improve the quality of life of people at risk of developing MS depend on the implementation of educational strategies aimed at the different syndrome aspects (Mattos et al., 2021 Mattos, S., Moreira, T., Florêncio, R., & Cestari, V. (2021). Elaboração e validação de um instrumento para mensurar autopercepção de saúde em adultos. [Development and validation of an instrument to measure self-perceived health in adults] Saúde Em Debate, 45 (129), 366–377. https://doi.org/10.1590/0103-1104202112909
https://doi.org/10.1590/0103-11042021129...
; Świątkiewicz et al., 2021 Świątkiewicz, I., Woźniak, A., &Taub, P. R. (2021). Time-restricted eating & Metabolic Syndrome: Current status and future perspectives. Nutrients, 13(1), 221. https://doi.org/10.3390/nu13010221 .
https://doi.org/10.3390/nu13010221...
). Among the main limitations of health promotion strategies based on health education, the highlight is the difficulty in measuring individuals’ knowledge gain after educational interventions. Therefore, providing a validated instrument to analyze people’s knowledge on MS could contribute to the evolution of health education strategies aimed at lifestyle changes, as previously pointed out (Coluci et al., 2015 Coluci, M. Z. O., Alexandre, N. M. C., & Milani, D. (2015). Construção de instrumentos de medida na área da saúde.[Construction of measurement instruments in the area of health] Ciência & Saúde Coletiva, 20(3), 925–936. https://doi.org/10.1590/1413-81232015203.04332013
https://doi.org/10.1590/1413-81232015203...
). The instrument developed in this study was validated using standardized statistical criteria and can, therefore, be useful in analyzing the effectiveness of interventions aimed at increasing people’s level of knowledge on MS.

Studies conducted to evaluate the knowledge of people in different age groups state that the more a person learns about their own clinical condition, the better the effect of the treatment in relation to remission and recovery of their health (Anand et al., 2023 Anand, V. V., Goh, R. S. J., Nah, B., Koh, S. W. C., Lim, J., Neo, N. W. S., Chew, J., Lee, Y. Y., Chin, Y. H., Chong, B., Kong, G., Tan, B., Low, Z., Khoo, C. M., Goh, L. H., Loh, P. H., Chai, P., Dalakoti, M., Chan, M., … Chew, N. W. S. (2023). General public’s knowledge, awareness, and perception of cardiometabolic diseases: Data from a Singapore study population. Frontiers in Medicine, 10, 1193829. https://doi.org/10.3389/fmed.2023.1193829
https://doi.org/10.3389/fmed.2023.119382...
; Emiral et al., 2021 Emiral, G. O., Tozun, M., Atalay, B. I., Goktas, S., Dagtekin, G., Aygar, H., Arslantas, D., Unsal, A., Babaoglu, A. B., & Tirpan, K. (2021). Assessment of knowledge of metabolic syndrome and health literacy level among adults in Western Turkey. Nigerian Journal of Clinical Practice, 24(1), 28–37. https://doi.org/10.4103/njcp.njcp_88_18
https://doi.org/10.4103/njcp.njcp_88_18...
; Wang et al., 2019 Wang, Q., Chair, S. Y., Wong, E. M., Taylor-Piliae, R. E., Qiu, X. C. H., & Li, X. M. (2019). Metabolic Syndrome Knowledge among adults with cardiometabolic risk factors: A cross-sectional study. International Journal of Environmental Research and Public Health, 16(1), 159. https://doi.org/10.3390/ijerph16010159
https://doi.org/10.3390/ijerph16010159...
). Therefore, those who acquire knowledge and information about MS are more likely to follow a regular physical activity routine and maintain a healthy diet, thus presenting weight control, an important factor for the MS treatment (Wang et al., 2019 Wang, Q., Chair, S. Y., Wong, E. M., Taylor-Piliae, R. E., Qiu, X. C. H., & Li, X. M. (2019). Metabolic Syndrome Knowledge among adults with cardiometabolic risk factors: A cross-sectional study. International Journal of Environmental Research and Public Health, 16(1), 159. https://doi.org/10.3390/ijerph16010159
https://doi.org/10.3390/ijerph16010159...
). Thus, the use of instruments that contribute to assessing people’s knowledge on MS can provide early detection and intervention of risk factors, and educational settings (schools and universities) are the best places for the early implementation of such interventions (Yahia et al., 2014 Yahia, N., Brown, C., Rapley, M., & Chung, M. (2014). Assessment of college students’ awareness and knowledge about conditions relevant to metabolic syndrome. Diabetology & Metabolic Syndrome, 6, 111. https://doi.org/10.1186/1758-5996-6-111
https://doi.org/10.1186/1758-5996-6-111...
). Risk perception is an important factor for an effective change of habits, behaviors and even of a lifestyle (Cortez et al., 2018 Cortez, R. M. A., Batista, A. M. O., Rocha, M. R., Santos, I. M. S., Moura, J. R. A., & Silva, A. R. V. (2018). Students’ knowledge of metabolic syndrome after educational intervention. Revista Brasileira de Enfermagem, 71(Suppl 4), 1493-1499. http://dx.doi.org/10.1590/0034-7167-2017-0342
http://dx.doi.org/10.1590/0034-7167-2017...
; Yahia et al., 2014 Yahia, N., Brown, C., Rapley, M., & Chung, M. (2014). Assessment of college students’ awareness and knowledge about conditions relevant to metabolic syndrome. Diabetology & Metabolic Syndrome, 6, 111. https://doi.org/10.1186/1758-5996-6-111
https://doi.org/10.1186/1758-5996-6-111...
). In view of this, interventions that aim at health education, with information about MS, if well targeted, can promote transformation and emancipation, leading people to make more conscious choices (Cortez et al., 2018 Cortez, R. M. A., Batista, A. M. O., Rocha, M. R., Santos, I. M. S., Moura, J. R. A., & Silva, A. R. V. (2018). Students’ knowledge of metabolic syndrome after educational intervention. Revista Brasileira de Enfermagem, 71(Suppl 4), 1493-1499. http://dx.doi.org/10.1590/0034-7167-2017-0342
http://dx.doi.org/10.1590/0034-7167-2017...
).

Study Limitations

Although in statistical terms the sample composed of 600 participants was sufficient for the analyses, one cannot ignore the fact that a convenience sampling was employed and, therefore, must be considered as a limiting factor. Furthermore, note that there was no phase to measure the construct validity among the target audience before data collection, which also represents a study limitation. Together, these limitations bring some methodological questions to the validation process, which must be overcome in future instrument adaptation. As this is the first attempt to develop such an instrument, another point that must be considered in future studies is its adequacy in terms of the item content so that it can be used considering the three domains previously considered for this study.

The instrument is easy to apply and has, in its proposal, 34 statements, whose sum of scores can indicate people’s level of knowledge on MS, with higher scores indicating good knowledge of relevant aspects of MS prevention and treatment, and smaller ones indicating a limited knowledge. The version developed here obtained adequate psychometric qualities and, therefore, can be used to analyze the effectiveness of educational strategies aimed at MS. Future studies should evaluate the factorial structure obtained in this study in different samples to prove its dimensionality.

References

  • American Educational Research Association, American Psychological Association, & National Council on Measurement in Education (Eds.). (2014). Standards for educational and psychological testing. American Educational Research Association.
  • Ambrosini, S., Mohammed, S. A., Lüscher, T. F., Costantino, S., & Paneni, F. (2020). New mechanisms of vascular dysfunction in cardiometabolic patients: Focus on epigenetics. High Blood Pressure & Cardiovascular Prevention, 27(5), 363–371. https://doi.org/10.1007/s40292-020-00400-2
    » https://doi.org/10.1007/s40292-020-00400-2
  • Amirfaiz, S., & Shahril, M. R. (2019). Objectively measured physical activity, sedentary behavior, & metabolic syndrome in adults: Systematic review of observational evidence. Metabolic Syndrome and Related Disorders, 17(1), 1–21. https://doi.org/10.1089/met.2018.0032
    » https://doi.org/10.1089/met.2018.0032
  • Anand, V. V., Goh, R. S. J., Nah, B., Koh, S. W. C., Lim, J., Neo, N. W. S., Chew, J., Lee, Y. Y., Chin, Y. H., Chong, B., Kong, G., Tan, B., Low, Z., Khoo, C. M., Goh, L. H., Loh, P. H., Chai, P., Dalakoti, M., Chan, M., … Chew, N. W. S. (2023). General public’s knowledge, awareness, and perception of cardiometabolic diseases: Data from a Singapore study population. Frontiers in Medicine, 10, 1193829. https://doi.org/10.3389/fmed.2023.1193829
    » https://doi.org/10.3389/fmed.2023.1193829
  • Cano, S. J., & Hobart, J. C. (2011). The problem with health measurement. Patient Preference and Adherence, 5, 279-290. https://doi.org/10.2147/PPA.S14399
    » https://doi.org/10.2147/PPA.S14399
  • Cassepp-Borges, V., Balbinotti, M. A. A., & Teodoro, M. L. M. (2010). Tradução e validação de conteúdo: Uma proposta para a adaptação de instrumentos. In: L. Pasquali. Instrumentação psicológica [Translation and Content Validation: A proposal for adapting instruments. In: L. Pasquali. Psychological instrumentation] (pp. 506-520). Artmed.
  • Coluci, M. Z. O., Alexandre, N. M. C., & Milani, D. (2015). Construção de instrumentos de medida na área da saúde.[Construction of measurement instruments in the area of health] Ciência & Saúde Coletiva, 20(3), 925–936. https://doi.org/10.1590/1413-81232015203.04332013
    » https://doi.org/10.1590/1413-81232015203.04332013
  • Cortez, R. M. A., Batista, A. M. O., Rocha, M. R., Santos, I. M. S., Moura, J. R. A., & Silva, A. R. V. (2018). Students’ knowledge of metabolic syndrome after educational intervention. Revista Brasileira de Enfermagem, 71(Suppl 4), 1493-1499. http://dx.doi.org/10.1590/0034-7167-2017-0342
    » https://doi.org/10.1590/0034-7167-2017-0342
  • Elangovan, N., & Sundaravel, N. (2021). Method of preparing a document for survey instrument validation by experts. MethodsX, 8, 101326. https://doi.org/10.1016/j.mex.2021.101326
    » https://doi.org/10.1016/j.mex.2021.101326
  • Emiral, G. O., Tozun, M., Atalay, B. I., Goktas, S., Dagtekin, G., Aygar, H., Arslantas, D., Unsal, A., Babaoglu, A. B., & Tirpan, K. (2021). Assessment of knowledge of metabolic syndrome and health literacy level among adults in Western Turkey. Nigerian Journal of Clinical Practice, 24(1), 28–37. https://doi.org/10.4103/njcp.njcp_88_18
    » https://doi.org/10.4103/njcp.njcp_88_18
  • Ferret, J. C. F., Yamaguchi, M. U., Branco, B. H. M., & Bernuci, M. P. (2021). Interventions based on practice of resistance exercises: A systematic review. Journal of Physical Education and Sport, 21(4), 1705-1714. https://doi.org/10.7752/jpes.2021.04216
    » https://doi.org/10.7752/jpes.2021.04216
  • Hernández-Nieto, R. A. (2002). Contribuciones al análisis estadístico de datos. [Contributions to statistical data analysis.] Booksurge.
  • Jaarsma, T., Westland, H., Vellone, E., Freedland, K. E., Schröder, C., Trappenburg, J. C. A., Stromberg, A., & Riegel, B. (2020). Status of theory use in self-care research. International Journal of Environmental Research and Public Health, 17(24), 9480. https://doi.org/10.3390/ijerph17249480
    » https://doi.org/10.3390/ijerph17249480
  • Leite, S. S., Áfio, A. C. E.; Carvalho, L. V.; Silva, J. M.; Almeida, P. C., & Pagliuca, L. M. F. (2018). Construction and validation of an educational content validation instrument in health. Revista Brasileira de Enfermagem, 71(Suppl 4), 1635–1641. https://doi.org/10.1590/0034-7167-2017-0648 .
    » https://doi.org/10.1590/0034-7167-2017-0648
  • Lim, S., & Jahng, S.(2019). Determining the number of factors using parallel analysis and its recent variants. Psychological Methods, 24(4), 452–467. https://doi.org/10.1037/met0000230
    » https://doi.org/10.1037/met0000230
  • Luoma, J. B., O’Hair, A. K., Kohlenberg, B. S., Hayes, S. C., & Fletcher, L. (2010). The development and psychometric properties of a new measure of perceived stigma toward substance users. Substance Use & Misuse, 45(1–2), 47–57. https://doi.org/10.3109/10826080902864712
    » https://doi.org/10.3109/10826080902864712
  • Mahadzir, M. D. A., Quek, K. F., & Ramadas, A. (2020). Nutrition and lifestyle behavior peer support program for adults with Metabolic Syndrome: Outcomes and lessons learned from a feasibility trial. Nutrients, 12(4), 1091. https://doi.org/10.3390/nu12041091
    » https://doi.org/10.3390/nu12041091
  • Malacarne, M. P., Luiz, S. G., Amaral, T. R., & Siqueira, M. M. (2017). Avaliação de serviços de saúde no campo da saúde coletiva: Uma sistematização em curso [Evaluation of health services in the field of collective health: An ongoing systematization]. Revista Brasileira de Pesquisa Em Saúde, 18(1), 62–67. https://doi.org/10.21722/rbps.v18i1.15136
    » https://doi.org/10.21722/rbps.v18i1.15136
  • Mattos, S., Moreira, T., Florêncio, R., & Cestari, V. (2021). Elaboração e validação de um instrumento para mensurar autopercepção de saúde em adultos. [Development and validation of an instrument to measure self-perceived health in adults] Saúde Em Debate, 45 (129), 366–377. https://doi.org/10.1590/0103-1104202112909
    » https://doi.org/10.1590/0103-1104202112909
  • Nilson, E. A. F., Andrade, R. C. S., Brito, D. A., & Oliveira, M. L. (2020). Custos atribuíveis a obesidade, hipertensão e diabetes no Sistema Único de Saúde, Brasil, 2018. [Costs attributable to obesity, hypertension and diabetes in the Unified Health System, Brazil, 2018] Revista Panamericana de Salud Pública, 44, e32. https://doi.org/10.26633/RPSP.2020.32
    » https://doi.org/10.26633/RPSP.2020.32
  • Nutbeam, D., & Muscat, D. M. (2021). Health Promotion Glossary 2021. Health Promotion International, 36(6), 1457-1598. https://doi.org/10.1093/heapro/daaa157
    » https://doi.org/10.1093/heapro/daaa157
  • Rattray, J., & Jones, M. C. (2007). Essential elements of questionnaire design and development. Journal of Clinical Nursing, 16(2), 234–243. https://doi.org/10.1111/j.1365-2702.2006.01573.x
    » https://doi.org/10.1111/j.1365-2702.2006.01573.x
  • Rodríguez de Vera, L., López Martínez, A. B., & Muria, A. J. (2021). Validação por expertos de uma proposta metodológica para promoção de atitudes positivas perante a deficiência em ambientes inclusivos. [Validation by experts of a methodological proposal to promote positive attitudes towards disability in inclusive environments.] Cuadernos de Psicología Del Deporte, 21(1), 162–178. https://doi.org/10.6018/cpd.436261
    » https://doi.org/10.6018/cpd.436261
  • Semnani-Azad, Z., Khan, T. A., Blanco Mejia, S., Souza, R. J., Leiter, L. A., Kendall, C. W. C., Hanley, A. J., & Sievenpiper, J. L. (2020). Association of major food sources of fructose-containing sugars with incident metabolic syndrome. JAMA Network Open, 3(7), e209993. https://doi.org/10.1001/jamanetworkopen.2020.9993
    » https://doi.org/10.1001/jamanetworkopen.2020.9993
  • Świątkiewicz, I., Woźniak, A., &Taub, P. R. (2021). Time-restricted eating & Metabolic Syndrome: Current status and future perspectives. Nutrients, 13(1), 221. https://doi.org/10.3390/nu13010221 .
    » https://doi.org/10.3390/nu13010221
  • Tibúrcio, M. P., Melo, G. S. M., Balduíno, L. S. C., Freitas, C. C. S., Costa, I. K. F., & Torres, G. V. (2015). Content validation of an instrument to assess the knowledge about the measurement of blood pressure. Revista de Pesquisa Cuidado é Fundamental Online, 7(2), 2475–2485. https://doi.org/10.9789/2175-5361.2015.v7i2.2475-2485
    » https://doi.org/10.9789/2175-5361.2015.v7i2.2475-2485
  • Vieira, M. S. N.,Matias, K. K., & Queiroz, M. G. (2021). Educação em saúde na rede municipal de saúde: Práticas de nutricionistas. [Health education in the municipal health network: Practices of nutritionists.] Ciência & Saúde Coletiva, 26(2), 455–464. https://doi.org/10.1590/1413-81232021262.41062020
    » https://doi.org/10.1590/1413-81232021262.41062020
  • von Elm,E.,Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., Vandenbroucke, J. P., & STROBE Initiative (2008). The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: Guidelines for reporting observational studies. Journal of Clinical Epidemiology, 61(4), 344–349. https://doi.org/10.1016/j.jclinepi.2007.11.008
    » https://doi.org/10.1016/j.jclinepi.2007.11.008
  • Wang, Q., Chair, S. Y., Wong, E. M., Taylor-Piliae, R. E., Qiu, X. C. H., & Li, X. M. (2019). Metabolic Syndrome Knowledge among adults with cardiometabolic risk factors: A cross-sectional study. International Journal of Environmental Research and Public Health, 16(1), 159. https://doi.org/10.3390/ijerph16010159
    » https://doi.org/10.3390/ijerph16010159
  • Yahia, N., Brown, C., Rapley, M., & Chung, M. (2014). Assessment of college students’ awareness and knowledge about conditions relevant to metabolic syndrome. Diabetology & Metabolic Syndrome, 6, 111. https://doi.org/10.1186/1758-5996-6-111
    » https://doi.org/10.1186/1758-5996-6-111
  • Article derived from the doctoral dissertation of the first author under the supervision of the third and fourth, defended in 2022, at the Graduate Program in Health Promotion of Universidade Cesumar.
  • How to cite this article:
    Ferret, J. C. F., Oliveira, L. P., Branco, B. H. M., & Bernuci, M. P. (2024). Evidence of validity of the knowledge assessment instrument on metabolic syndrome. Paidéia (Ribeirão Preto), 34, e3418. doi: https://doi.org/10.1590/1982-4327e3418

Edited by

Associate editor:
Clarissa Mendonça Corradi-Webster

Publication Dates

  • Publication in this collection
    13 Sept 2024
  • Date of issue
    2024

History

  • Received
    20 Apr 2022
  • Accepted
    13 May 2024
  • Reviewed
    16 Sept 2022
  • Reviewed
    16 Oct 2023
  • Reviewed
    25 Jan 2024
Universidade de São Paulo, Faculdade de Filosofia Ciências e Letras de Ribeirão Preto, Programa de Pós-Graduação em Psicologia Av.Bandeirantes 3900 - Monte Alegre, 14040-901 Ribeirão Preto, Tel.: (55 16) 3315-3829 - Ribeirão Preto - SP - Brazil
E-mail: paideia@usp.br