Evaluating the psychometric properties of the eHealth Literacy Scale in Brazilian adults

Evaluación de las propiedades psicométricas del instrumento eHealth Literacy Scale en adultos brasileños

ABSTRACT

Objective:  To translate and adapt the eHealth Literacy Scale for the cultural reality of Brazil and to evaluate the psychometric properties of its Brazilian Portuguese version.

Methods:  The instrument was translated and adapted to Brazilian Portuguese and applied to a sample of 502 individuals from 18 to 80 years old who lived in the surrounding areas of six Family Health Units of a city in the countryside of the state of São Paulo, Brazil. The data was evaluated using exploratory and confirmatory factor analysis, item response theory, and instrument reliability measures (Cronbach’s alpha and McDonald’s omega).

Results:  The eHealth Literacy Scale - Brazilian version (eHEALS-Br) presented an excellent internal consistency (α = 0.95 e ω = 0.95), with only one dimension and an explained variation of 81.79%.

Conclusions:  The Brazilian version of the instrument showed excellent psychometric properties to measure the levels of digital health literacy in adults from the country.

Descriptors: Health Literacy; Validation Studies; Computer Literacy; Unified Health System; Primary Health Care

RESUMEN

Objetivo:  Traducir y adaptar la eHealth Literacy Scale a realidad cultural Brasileña y evaluar sus propiedades psicométricas de la versión en portugués brasileño.

Métodos:  El instrumento fue traducido y adaptado al portugués brasileño y, en seguida, aplicado en una muestra de 502 individuos entre 18 y 80 años residentes en áreas cercas a seis Unidades de Salud de la Familia de un municipio del interior del estado de São Paulo, Brasil. Los datos fueron evaluados mediante análisis factorial exploratoria y confirmatoria, Teoría de Respuesta al Ítem y confiabilidad del instrumento (alfa de Cronbach y omega de McDonald).

Resultados:  El instrumento eHealth Literacy Scale - versión brasileña (eHEALS-Br) presentó excelente consistencia interna (α = 0,95 y ω = 0,95), apenas una dimensión y variancia explicada de 81,79%.

Conclusiones:  La versión brasileña del instrumento mostró excelentes propiedades psicométricas para contraste de los niveles de alfabetización digital en salud en adultos del nuestro país.

Descriptores: Alfabetización en Salud; Estudios de Validación; Alfabetización Digital; Sistema Único de Salud; Atención Primaria de Salud

RESUMO

Objetivo:  Traduzir e adaptar a eHealth Literacy Scale para a realidade cultural do Brasil e avaliar suas propriedades psicométricas da versão em português brasileiro.

Métodos:  O instrumento foi traduzido e adaptado ao português brasileiro e, em seguida, aplicado em uma amostra de 502 indivíduos entre 18 e 80 anos residentes em áreas circunvizinhas a seis Unidades de Saúde da Família de um município do interior do estado de São Paulo, Brasil. Os dados foram avaliados mediante análises fatorial exploratória e confirmatória, Teoria de Resposta ao Item e confiabilidade do instrumento (alfa de Cronbach e ômega de McDonald).

Resultados:  O instrumento eHealth Literacy Scale - versão brasileira (eHEALS-Br) apresentou excelente consistência interna (α = 0,95 e ω = 0,95), apenas uma dimensão e variância explicada de 81,79%.

Conclusões:  A versão brasileira do instrumento mostrou excelentes propriedades psicométricas para aferição dos níveis de letramento digital em saúde em adultos do nosso país.

Descritores: Letramento em Saúde; Estudos de Validação; Alfabetização Digital; Sistema Único de Saúde; Atenção Primária à Saúde

INTRODUCTION

Health literacy is seen by the World Health Organization (WHO) as an important social health determinant⁽¹⁾. It is a construct which considers people’s knowledge, confidence, and skills to access, understand, judge, and apply the information in their process of decision making in health⁽¹⁾.

The transmission of health information through the Internet has been a worldwide trend, making it the main media for health communication⁽²⁾, and there is a wide variety of information available online^(3-4). In the Brazilian context, the percentage of residences that use the Internet increased, between 2016 and 2017, from 69.3% to 74.9%, that is, three in every four Brazilian households acquire information using this source⁽⁵⁾, including information related to health^(3,6).

Internet enables individuals to access, at any time and place, information about health⁽⁴⁾. However, due to this ease of access, much of the information made available online does not go through any quality assurance process⁽⁴⁾. This, instead of aiding in assertive decision making, leads to uninformed individuals and community, due to the excess of information and to the difficulty to evaluate its quality^(3-4,7-8).

In this context, the field of eHealth (Electronic Health) emerged, which concerns the use of information and communication technologies for health⁽³⁾. This ability is necessary for individuals to evaluate the quality of the information displayed on the Internet to make assertive health decisions, that is, it is necessary for them to have adequate eHealth literacy.

The eHealth literacy is defined by Norman and Skinner⁽⁹⁾ as “the ability to search, find, understand, and evaluate health information from electronic sources and apply the knowledge acquired to address or deal with health problems”. The authors proposed, to measure this construct, the eHealth Literacy Scale (eHEALS)⁽⁹⁾.

The eHEALS scale was one of the first developed to measure the level of digital literacy in health. It includes eight items that aim to measure knowledge, comfort, and perceived skills of individuals at finding, evaluating, and applying electronic health information to health problems⁽⁹⁾.

The scale was originally developed in English⁽⁹⁾ and validated in many populations, including adolescents^(9-11), university students⁽¹²⁾, adults^(13-15), and the elderly^16-17) This is the main measure used to evaluate digital health literacy, and it is validated in the languages European Portuguese⁽¹¹⁾, Spanish⁽¹²⁾, Korean⁽¹³⁾, German⁽¹⁴⁾, Polish^(15)), Chinese⁽¹⁸⁾, Japanese⁽¹⁹⁾, Italian⁽²⁰⁾, Hungarian⁽²¹⁾, Serbian⁽²²⁾, Amharic (Ethiopia)⁽²³⁾, Swedish⁽²⁴⁾, and Greek⁽²⁵⁾. The scale showed good psychometric properties in all these languages.

Furthermore, it has been applied o populations such as adults with chronic diseases⁽²⁶⁾, individuals with HIV⁽²⁷⁾, and otorhinolaryngology ⁽²⁸⁾. However, so far, no instrument of the sort has been validated for the Brazilian population.

Therefore, efforts should be made to make available a tool that can explore and explain the structure and function of digital health literacy to instrumentalize health teams to monitor the efficacy and equity of interventions carried out digitally and indicate the impact of the use of digital means in the making of health decisions, to contribute for the adaptation of care strategies that use this resource.

OBJECTIVE

To translate and adapt the eHealth Literacy Scale for the cultural reality of Brazil and to evaluate the psychometric properties of its Brazilian Portuguese version.

METHODS

Ethical aspects

The research project was approved by the Research Ethics Committee of a teaching institution according to the recommendations of Resolution 466/2012 from the National Council of Health. All participants signed the Free and Informed Consent Form.

Design, period, and place of study

A cross-sectional study was carried out with individuals who lived in the nearby areas of six Family Health Units (USF) located in a medium-sized city in the countryside of the state of São Paulo. The study was carried out from March to October 2019.

Population or sample; criteria of inclusion and exclusion

The population of the study was formed by individuals from 18 to 80 years old, who lived in the surroundings of six USFs. Those who presented mental and/or cognitive problems, with a medical diagnosis from the USF that stated their inability to respond to the instruments, were excluded. The pre-test was carried out with 50 individuals who lived near one of the USFs. After the instrument was filled in with pen and paper through a self-application, the researcher read the entire instrument with them, to search for difficulties of the respondents in comprehending specific words, questions, and/or responses. In the test stage of the final version of the instrument, it was also self-applied (like the original study) in a sample that included 502 individuals from six USFs, with a mean of 80-90 per USF. In both moments, the participants were randomly selected, had from 18 to 80 years old, and were addressed in their residences for data collection.

Instrument

The instrument is a scale with eight Likert items that varied from 1 (entirely disagree) to 5 (entirely agree), and its total score could vary from 8 to 40. The higher the score, the higher the level of digital health literacy⁽⁹⁾. There were also two introductory eHEALS questions, which are: “How much do you think internet is useful to help you in making health decisions?” (not useful, little useful, I am not sure, useful, very useful) and “How important is it for you to access the health information/resources available on the Internet?” (not important, little important, I am not sure, important, very important).

Procedures

Before the study started, the main authors of the original study were asked⁽⁹⁾, via e-mail, for their permission to translate and carry out a transcultural adaptation of the instrument, following international recommendations^(29-30): (1) translation into Portuguese with semantic, idiomatic, and conceptual equivalence; (2) back-translation by qualified professionals; (3) specialist committee for the multidisciplinary revision of all translations and back-translations; (4) pre-test to evaluate the equivalence; (5) adjustments when needed. The original eHEALS scale was translated from English to Brazilian Portuguese by two English teachers: one of them had knowledge about the research, while another was a researcher from the health field with knowledge about the English language. The version that found a consensus was, then, translated back into English by two translators whose native language was English and who did not participate in the first stage of the translation.

Then, a specialist committee was formed by six professionals from the field of health who had experiences in the field of health literacy and a high level of proficiency in the English language. Their objective was to evaluate the process and propose a final version for the document. The main author of the original instrument was contacted to clarify any doubts about the meaning of some issues and to propose modifications, to generate an instrument whose semantic validity was adequate to our reality. All questions from the instrument were found to be valid by the researchers to measure the construct. When the translated versions were compared, there were small differences in the translation of the title (“electronic literacy in health” vs. “digital literacy in health “) and in some words from the first and second questions, which did not change the meaning of the sentences and were equalized by consensus after a meeting. Translation of only one item of the instrument raised doubt: health resources. After the original author was contacted, the researchers decided to include a sentence in the disclaimer of the instrument, explaining the meaning of the term. This was the only change in the instrument that was more relevant.

In the pre-test stage, there was no need for changes in the instrument, since less than 10% of interviewees reported doubts about any item⁽³⁰⁾.

Analysis of results and statistics

Contemporary psychometry, especially after the concept of evidence of validity⁽³¹⁾ has required extensive testing and the integration of many stages in the several stages of instrument validation. Therefore, our analysis was based on this concept, with multiple indicators in the search for evidences of the internal structure validity, integration the three most used techniques in this stage: exploratory factor analysis (EFA - unrestricted model); confirmatory factor analysis (CFA - restricted model); and item response theory (IRT).

The dimensionality of the instrument was tested using the robust parallel analysis (RPA) through the optimal implementation of parallel analysis(PA) with the minimum rank factor analysis, which minimizes the common residue variance⁽³²⁾. The robustness of the test was verified by associating a bootstrap and extrapolating the sample to 5,000. The polychoric matrix was estimated using Bayes modal estimation⁽³³⁾. The dimensionality of the exploratory factor analysis (unrestricted model) was tested by a parallel analysis, which is considered to be one of the most robust and precise techniques to test dimensionality⁽³⁴⁾. Factors were extracted using the RULS technique (Robust Unweighted Least Squares), which reduces the residue from the matrixes⁽³⁵⁾ and is more robust for non-normal data⁽³⁶⁾. If the instrument is found to be multidimensional, the Promax rotation is used, which is a non-orthogonal⁽³⁷⁾ technique, the most appropriate for latent psychosocial variables⁽³⁸⁾. In addition, the following were adopted as indicators to evaluate unidimensionality⁽³⁹⁾: UNICO (Unidimensional Congruence > 0.95), ECV (Explained Common Variance > 0.80), and MIREAL (Mean of Item Residual Absolute Loadings < 0.30).

Validation techniques are recommended to increase their precision and the quality of their instruments⁽⁴⁰⁾, bringing more influence to the model⁽⁴¹⁾. Consequently, the technique Normal-Ogive Graded Response Model⁽⁴²⁾ was used to evaluate the adjustment of factor loadings. The index of discrimination of item (a) was adopted to corroborate the exploratory factor analysis, since it measures the strength of the association between the item and the latent variable⁽⁴³⁾ and its interpretation is similar to the factor loadings from the EFA⁽⁴⁴⁾. This study followed the recommendation according to which that an a below 0.65 is considered to have low discriminatory power; with 0.65 to 1.34 indicating moderate discriminatory power; from 1.35 to 1.69 to indicate a high discriminatory power; while an a above 1.70 was found to indicate a very high discriminatory power⁽⁴⁵⁾.

Regarding the quality parameters of the instruments, the explained variation of the instrument must be around 60%⁽³⁸⁾. The factor loadings of 0.30 are recommended when the sample has at least 300 individuals⁽³⁸⁾, but the model is recommended to search factor loadings above 0.50⁽⁴⁶⁾; the commonalities must have values above 0.40⁽⁴⁶⁾. Keeping or removing an item in the model depends on the magnitude of the commonalities, of factor loadings, on the size of the sample and on the degree to which the item manages to measure the factor of the nonexistence of cross-loading and Heywood cases. The reliability of the instrument was evaluated using the indicators Cronbach’s alpha ⁽⁴⁷⁾ and McDonald’s omega (ω)⁽⁴⁸⁾. Two indicators were adopted to increase the reliability of the interpretation. Values above ≥ 0.7 in the reliability indexes have been considered to be adequate⁽⁴⁹⁾.

The factor loadings and the predictive power of the item (R²) were considered to be fitness indexes for the confirmatory factor analysis. For the goodness-of-fit, a robust mean and variance-adjusted chi square was used. Factor loadings above 0.50 and minimal fitness indexes, considering the number of participants and variables, were: (Non-Normed Fit Index) > 0.95; CFI (Comparative Fit Index) > 0.95; GFI (Goodness Fit Index) > 0.95; AGFI (Adjusted Goodness Fit Index) > 0.95; RMSEA (Root Mean Square Error of Approximation) < 0.08 and the RMSR (Root Mean Square of Residuals) < 0.8.

The replicability of the construct was evaluated by the Generalized G-H Index with an index above 0.80. The quality and effectiveness of the estimates of the factors were evaluated using the Factor Determinacy Index (recommended for adequate estimates with values above 0.90), EAP marginal reliability > 0.80, sensibility ratio (SR) > 2, and Expected Percentage of True Differences (EPTD) > 90%. These complementary indexes were used because the evaluation of primary indexes(goodness-of-fit) in itself does not guarantee that the solution for the factor analysis will be good or useful in practice, since satisfactory solution indexes can be obtained from low-quality items⁽³⁹⁾. The analysis were carried out using the SPSS 23, AMOS 23, and Factor 10.10.1.

RESULTS

The mean age of the participants was 39.3 years old (±13.3). Only 9.3% were above 60 years old, and 65.3% were females. Additionally, 255 (50.8%) individuals had family income of up to two minimum wages. 402 (80.1%) had completed at least elementary school. Regarding how useful respondents thought internet was, 49.6% stated useful/very useful, while 24.3% were uncertain. Regarding the importance attributed by people to the ability to access health information/resources on the Internet, 52.6% thought it was important/very important, and 22.1% were uncertain.

Chart 1 shows the content of the final adapted version of the eHEALS-Br scale.

Chart 1
Final adapted version of the eHEALS-Br

I would like to know your opinion and your experience with the use of the internet to search for information about health, according to the questions below. For each statement, mark the answer that best reflects your opinion and your current experience. I would also like to clarify that the term "health resources", in the questions below, refers to both websites and to applications related to health.

I know which health resources are available on the Internet. 1) … Entirely disagree    2) … partially disagree    3) … I am not sure    4) … Partially agree    5) … Entirely agree I know where to find useful health resources on the internet. 1) … Entirely disagree    2) … partially disagree    3) … I am not sure    4) … Partially agree    5) … Entirely agree I know how to find useful health resources on the internet. 1) … Entirely disagree    2) … partially disagree    3) … I am not sure    4) … Partially agree    5) … Entirely agree I know how to use the Internet to clarify doubts about health. 1) … Entirely disagree    2) … partially disagree    3) … I am not sure    4) … Partially agree    5) … Entirely agree I know how to use the information about health I find on the internet to help me. 1) … Entirely disagree    2) … partially disagree    3) … I am not sure    4) … Partially agree    5) Entirely agree 5) … Entirely agree I have the skills I need to evaluate the health resources I find on the internet. 1) …Entirely disagree    2)… partially disagree    3) … I am not sure    4) … Partially agree    5)… Entirely agree I can perceive which health resources have a high quality and which have low quality on the Internet. 1) …Entirely disagree    2)… partially disagree    3) … I am not sure    4) … Partially agree    5)… Entirely agree I feel safe using information from the Internet to make health-related decisions. 1) …Entirely disagree    2)… partially disagree    3) … I am not sure    4) … Partially agree    5)… Entirely agree

Table 1 shows the mean values for each item of the instrument eHealth Literacy Scale - Brazilian scale (eHEALS-Br).

Table 1
Score (mean with standard deviation) of the interviewees in each item of the instrument eHEALS - Brazilian version (eHEALS-Br)

eHEALS-Br items		Mean (SD)
Item 1	I know which health resources are available on the Internet.	3.18 (1.18)
Item 2	I know where to find useful health resources on the internet.	3.25 (1.20)
Item 3	I know how to find useful health resources on the internet.	3.35 (1.19)
Item 4	I know how to use the Internet to clarify doubts about health.	3.45 (1.19)
Item 5	I know how to use the information about health I find on the internet to help me.	3.46 (1.16)
Item 6	I have the skills I need to evaluate the health resources I find on the internet.	3.17 (1.22)
Item 7	I can perceive which health resources have a high quality and which have low quality on the Internet.	3.21 (1.25)
Item 8	I feel safe using information from the Internet to make health-related decisions.	2.82 (1.23)

• Caption: eHEALS-Br - Brazilian version of the instrument eHealth Literacy Scale; SD - standard deviation.

It was found that the mean values varied from 2.82 (Item 8) to 3.46 (Item 5). The mean total score of the scale eHEALS-Br for the population evaluated was 25.1 (±8,1).

The fitness analysis of the sample for the factor analyses led to a matrix determinant of (0.00013), Kaiser Meyer and Olkin (0.90) and Bartlett’s test of sphericity (4443.7; p < 0.0001). The polychoric correlations of the items varied from 0.60 to 0.93. All indicators suggest the data is of good quality for the factor analysis.

The dimensionality calculated through APR showed only one dimensions with an eigenvalue of 5.86, leading to an explained variation of 81.79%. The unidimensionality of the model was confirmed by the values of UNICO (0.99), ECV (0.93), and MIREAL (0.21). As a result, it was not necessary to rotate the model.

The factor loadings settled between 0.75 and 0.90, with commonality in the range from 0.57 and 0.81 and discrimination of the item from 1.16 to 2.12, showing that the items measure the latent variable. The reliability analysis showed an alpha of 0.95 and an omega of 0.95. Furthermore, the GH index was below 0.80, indicate the stability of the model in other populations and sub-samples, with latent G-H of 0.96 and observed G-H of 0.90.

In Table 2 are presented the values of factor loadings, commonality, and item discrimination.

Table 2
Values of factor loadings, commonalities, and item discrimination of the questions of the eHEALS instrument, Brazilian version (eHEALS-Br)

Question	λ	h2	a
Q1	0.83	0.70	1.53
Q2	0.90	0.81	2.06
Q3	0.90	0.81	2.12
Q4	0.88	0.77	1.85
Q5	0.88	0.78	1.92
Q6	0.87	0.76	1.78
Q7	0.79	0.63	1.31
Q8	0.75	0.57	1.16

• λ = factor loading; h2 = commonalities; a = item discrimination

The index of the quality and effectiveness of score estimates also had adequate levels: Factor Determinacy Index (FDI) = 0.980, EAP marginal reliability = 0.961, sensitivity ratio (SR) = 4.955 and expected percentage of true differences (EPTD) = 96.1%. These indexes show that the score of the instrument is consistent and is not established by chance or randomly⁽⁴⁴⁾.

The confirmatory factor analysis found that factor loadings varied from 0.71 to 0.87, with a predictive value of the item (R2) from 0.68 to 0.87 (Fig. 1). In addition to primary indicators, the quality indexes of the model were: χ² = 93.17; p < 0.0001; NNFI = 0.98; CFI = 0.99; GFI = 0.99; AGFI = 0.99; RMSEA = 0.08; e RMSR = 0.05. In addition to the GOF fitness, the eigenvalue (8.25) considering the covariance also established that the model was unidimensional.

Figure 1
Pathway diagram for the questions in the Brazilian version of the eHealth Literacy Scale (eHEALS-Br)

DISCUSSION

The analyses have shown that the Brazilian Portuguese version of the eHEALS scale (eHEALSBr) was found to be adequate for all techniques (AFE, AFC, TRI) and indicators used. Although the sampling process of this study was not broad enough to represent all 18-year-old or older people from the city and from Brazil, it was found to have good psychometric properties to measure the digital literacy construct in Brazilian adults.

Concerning its reliability, the Brazilian version of the eHEALS instrument (eHEALSBr) showed high values in both criteria (α and ω = 0,95). Furthermore, the G-H index suggests that the model can be replicated to other populations and sub-samples, mitigating the potential effects of the characteristics of the sample. Although the alpha is not a good index to compare the models, it is the only common indicator between our study and others that tested the eHEALS. The values of α were higher than those found in the original study, which was carried out with adolescents in the United States (α = 0.88)⁽⁹⁾. The same was true for other studies with young populations, such as that of university students in Spain (α = 0.88)⁽¹²⁾, adolescents in Portugal (α = 0.84)⁽¹¹⁾, and young adults in Korea (α = 0.88)⁽¹³⁾. It is interesting to note that higher values in the instrument, closer to the ones found in this study, were also found in researches with older populations^(14,16,19).

The factor analysis showed that the instrument had unidimensional features, similar to those of the original document in English⁽⁹⁾, which was corroborated by versions in other languages^{(12-13,16,19-21)}. Some studies, however, found a two-factor structure^(10,20), while others found that the model was only found to be adequately fit when the score of the eHEALS instrument was adjusted in a three-factor model⁽²⁶⁾.

The unidimensional factor model had an explained variation of 81.79%, higher than found in the original Norman and Skinner study (56%)⁽⁹⁾. Other studies also found a variation from 47.80 to 75.81%^(11-16), although it was lower than the value found by Ma and Wu⁽¹⁸⁾ in a rural population in China (91.8%).

The mean total score of the instrument (25.1) was lower than that found in adults in Hungary (29.2)⁽²¹⁾ and similar to the result for elders in Poland (25.2)⁽¹⁵⁾. Corroborating the findings from other countries^(7,11,15,18), the eight question from the eHEALS-Br instrument had the highest mean among the items of the instrument, showing that people do not feel that safe when using information from the internet to make health-related decisions, when compared to other digital health literacy skills.

Although other tools to evaluate digital literacy in health have been proposed, and despite the fact that the eHEALS evaluates the perception of the individual about health information from electronic sources, as opposed to their observed skills of searching, finding, understanding, and evaluating this information, it is one of the most used instruments to evaluate this construct, due to its good psychometric properties and ease of application^(8,15). Furthermore, as far as we know, this is the first study that evaluated the psychometric properties of the instrument in Primary Health Care users.

Considering the scarcity of instruments that are specific for measuring the digital literacy in health in Portuguese, coupled with the potential of the Internet to enable the empowering of individuals and of the community^(3,8), the Brazilian version of the eHEALS instrument can be used as a tool to plan health actions in different contexts. It stands out that the situation of the Brazilian population in regard to digital literacy in health is still unknown. Therefore, it is important to apply instrument to measure the level of literacy, contributing for professionals to educate the patient with regard to the use of electronic sources according to their performance. Also, considering the worldwide situation in the COVID-19 pandemic, the digital literacy in health has been gaining relevance, since, with social isolation, many resort to electronic resources to monitor their patients. Teleconsultations became a common reality, but it is a challenge for both patients and professionals, since its success and a successful access to reliable electronic sources can be positive or negative, and demand participation from both the professional and the patient.

In this setting, the Digital Health Strategy for Brazil 2020-2028 ⁽⁵⁰⁾, recently published, shows the goal of expanding the Single Health System (SUS) to improve the health care offered to Brazilian people in this aspect. The National Network of Health Data, part of the Program Conecte SUS, should be established up to 2028 as a digital platform with innovation, information, and health services for the country, involving users, citizens, patients, communities, managers, professionals, and health organizations. Although this is not the focus of the document, the goal of the program requires the population to have proper digital literacy in health. Otherwise, their participation will be reduced, as will the benefits of this innovation. The instrument validated here can contribute to operationalize this strategy, providing periodical diagnoses of the digital literacy in health of those involved, with results that can be used to reprogram procedures.

As a result, this study brings important contributions for the development of future investigations related to the impact of this construct in health care. Furthermore, this is the first work about the e-HEALS instrument with a robust testing framework, integrating the techniques used in the internal structure evidence stage.

Study limitations

It stands out that the family income in the sample was relatively low. Therefore, it is reasonable to assume that the access of this population to Internet and electronic devices may be limited. Still, due to the cross-sectional nature of this study, it is impossible to provide causal inferences.

Contributions for the field of nursing, health, or public policies

Nursing workers are in a privileged position to use strategies of health communication that contribute for the users of the Single Health System to be safer and more autonomous. In a setting where the population is increasingly present on-line and there is a considerable dissemination of fake news, it is paramount for health workers to incorporate the use of tools that allow them to understand the impact of the use of the information made available on the internet about the behavior of patients.

CONCLUSIONS

The Brazilian version of the eHealth Literacy Scale (eHEALS-Br) showed excellent evidences of the validity of its internal structure for the assessment of the levels of digital literacy in health for adults in Brazil.

REFERENCES

• ¹ Kickbusch I, Pelikan JM, Apfel F, Tsouros A. Health literacy: the solid facts [Internet]. 2013 [cited 2021 Aug 02]. Available from: https://apps.who.int/iris/bitstream/handle/10665/128703/e96854.pdf
  » https://apps.who.int/iris/bitstream/handle/10665/128703/e96854.pdf
• ² Chaffey D. Global social media research summary [Internet]. 2018 [cited 2019 Feb 12]. Available from: https:// www.smartinsights.com/social-media-marketing/socialmedia-strategy/new-global-social-media-research
  » https:// www.smartinsights.com/social-media-marketing/socialmedia-strategy/new-global-social-media-research
• ³ Lopes MACQ, Oliveira GMM, Maia LM. Digital health, universal right, duty of the state? Arq Bras Cardiol. 2019;113(3):429-34. https://doi.org/10.5935/abc.20190161
  » https://doi.org/10.5935/abc.20190161
• ⁴ Jaks R, Baumann I, Juvalta S, Dratva J. Parental digital health information seeking behavior in Switzerland: a cross-sectional study. BMC Public Health. 2019;19(1):225. https://doi.org/10.1186/s12889-019-6524-8
  » https://doi.org/10.1186/s12889-019-6524-8
• ⁵ Instituto Brasileiro de Geografia e Estatística (IBGE). Diretoria de Pesquisas, Coordenação de Trabalho e Rendimento. Pesquisa Nacional por Amostra de Domicílios Contínua 2017 [Internet]. 2018 [cited 2019 Dec 20]. Available from: https://agenciadenoticias.ibge.gov.br/agencia-sala-de-imprensa/2013-agencia-de-noticias/releases/23445-pnad-continua-tic-2017-internet-chega-a-tres-em-cada-quatro-domicilios-do-pais
  » https://agenciadenoticias.ibge.gov.br/agencia-sala-de-imprensa/2013-agencia-de-noticias/releases/23445-pnad-continua-tic-2017-internet-chega-a-tres-em-cada-quatro-domicilios-do-pais
• ⁶ Brazilian Internet Steering Committee. 2016 ICT Households - Survey on the Use of Information and Communication Technologies in Brazilian Households [Internet]. 2017 [cited 2021 Aug 02]. Available from: https://cetic.br/publicacao/pesquisa-sobre-o-uso-das-tecnologias-de-informacao-e-comunicacao-nos-domicilios-brasileiros-tic-domicilios-2016/
  » https://cetic.br/publicacao/pesquisa-sobre-o-uso-das-tecnologias-de-informacao-e-comunicacao-nos-domicilios-brasileiros-tic-domicilios-2016/
• ⁷ Soellner R, Huber S, Reder M. The concept of eHealth literacy and its measurement: German translation of the eHEALS. J Media Psychol. 2014;26:29-38. https://doi.org/10.1027/1864-1105/a000104
  » https://doi.org/10.1027/1864-1105/a000104
• ⁸ Kim H, Xie B. Health literacy in the eHealth era: a systematic review of the literature. Patient Educ Couns. 2017;100(6):1073-82. https://doi.org/10.1016/j.pec.2017.01.015
  » https://doi.org/10.1016/j.pec.2017.01.015
• ⁹ Norman CD, Skinner HA. eHEALS: The eHealth Literacy Scale. J Med Internet Res. 2006;8(4):e27. https://doi.org/10.2196/jmir.8.4.e27
  » https://doi.org/10.2196/jmir.8.4.e27
• ¹⁰ Holch P, Marwood JR. EHealth Literacy in UK Teenagers and Young Adults: Exploration of Predictors and Factor Structure of the eHealth Literacy Scale (eHEALS). JMIR Form Res. 2020;4(9):e14450. https://doi.org/10.2196/14450
  » https://doi.org/10.2196/14450
• ¹¹ Tomás CC, Queirós PJP, Ferreira TJR. Análise das propriedades psicométricas da versão portuguesa de um instrumento de avaliação de e-Literacia em Saúde. Rev Enf Referência. 2014;4(2):19-28. https://doi.org/10.12707/RIV14004
  » https://doi.org/10.12707/RIV14004
• ¹² Pérez GA, Almagro BJ, Gómez AH, Gómez JIA. Validación de la escala EHEALTH LITERACY (EHEALS) en población universitária española. Rev Esp Salud Pública. 2015;89:329-38. https://doi.org/10.4321/S1135-57272015000300010
  » https://doi.org/10.4321/S1135-57272015000300010
• ¹³ Chung SY, Park BK, Nahm E-S. The Korean eHealth Literacy Scale (K-eHEALS):Reliability and Validity Testing in Younger Adults Recruited Online. J Med Internet Res [Internet]. 2018 [cited 2021 Mar 5]; 20(4):e138. Available from: https://www.jmir.org/2018/4/e138/
  » https://www.jmir.org/2018/4/e138/
• ¹⁴ van der Vaart R, van Deursen AJ, Drossaert CH, Taal E, van Dijk JA, van de Laar MA. Does the eHealth Literacy Scale (eHEALS) measure what it intends to measure? Validation of a Dutch version of the eHEALS in two adult populations. J Med Internet Res [Internet]. 2011 [cited 2021 Mar 6];13(4):e86. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3222202/
  » https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3222202/
• ¹⁵ Duplaga M, Sobecka K, Wójcik S. The Reliability and Validity of the Telephone-Based and Online Polish eHealth Literacy Scale Based on Two Nationally Representative Samples. Int J Environ Res Public Health [Internet]. 2019 [cited 2021 Mar 6];16(17):3216. Available from: https://www.mdpi.com/1660-4601/16/17/3216
  » https://www.mdpi.com/1660-4601/16/17/3216
• ¹⁶ Chung SY, Nahm ES. Testing reliability and validity of the eHealth Literacy Scale (eHEALS) for older adults recruited online. Comput Inform Nurs. 2015;33(4):150-6. https://doi.org/10.1097/CIN.0000000000000146
  » https://doi.org/10.1097/CIN.0000000000000146
• ¹⁷ Lin CY, Broström A, Griffiths MD, Pakpour AH. Psychometric Evaluation of the Persian eHealth Literacy Scale (eHEALS) Among Elder Iranians With Heart Failure. Eval Health Prof. 2020;43(4):222-9. https://doi.org/10.1177/0163278719827997
  » https://doi.org/10.1177/0163278719827997
• ¹⁸ Ma Z, Wu M. The Psychometric Properties of the Chinese eHealth Literacy Scale (C-eHEALS) in a Chinese Rural Population: Cross-Sectional Validation Study. J Med Internet Res [Internet]. 2019 [cited 2021 Mar 6];21(10):e15720. Available from: https://www.jmir.org/2019/10/e15720/
  » https://www.jmir.org/2019/10/e15720/
• ¹⁹ Mitsutake S, Shibata A, Ishii K, Okazaki K, Oka K. Developing Japanese version of the eHealth Literacy Scale (eHEALS). Nihon Koshu Eisei Zasshi [Internet]. 2011[cited 2021 Mar 6];58(5):361-71. Available from: https://pubmed.ncbi.nlm.nih.gov/21905612/
  » https://pubmed.ncbi.nlm.nih.gov/21905612/
• ²⁰ Diviani N, Dima AL, Schulz PJ. A Psychometric Analysis of the Italian Version of the eHealth Literacy Scale Using Item Response and Classical Test Theory Methods. J Med Internet Res [Internet]. 2017[cited 2021 Mar 6];19(4):e114. Available from: https://www.jmir.org/2017/4/e114/
  » https://www.jmir.org/2017/4/e114/
• ²¹ Zrubka Z, Hajdu O, Rencz F, Baji P, Gulácsi L, Péntek M. Psychometric properties of the Hungarian version of the eHealth Literacy Scale. Eur J Health Econ. 2019;20(Suppl 1):57-69. https://doi.org/10.1007/s10198-019-01062-1
  » https://doi.org/10.1007/s10198-019-01062-1
• ²² Gazibara T, Cakic J, Cakic M, Pekmezovic T, Grgurevic A. eHealth and adolescents in Serbia: psychometric properties of eHeals questionnaire and contributing factors to better online health literacy. Health Promot Int. 2019;34(4):770-778. https://doi.org/10.1093/heapro/day028
  » https://doi.org/10.1093/heapro/day028
• ²³ Shiferaw KB. Validation of the Ethiopian Version of eHealth Literacy Scale (ET-eHEALS) in a Population with Chronic Disease. Risk Manag Healthc Policy. 2020;13:465-471. https://doi.org/10.2147/RMHP.S240829
  » https://doi.org/10.2147/RMHP.S240829
• ²⁴ Wangdahl J, Jaensson M, Dahlberg K, Nilsson U. The Swedish Version of the Electronic Health Literacy Scale: Prospective Psychometric Evaluation Study Including Thresholds Levels. JMIR Mhealth Uhealth [Internet]. 2020 [cited 2021 Mar 6];8(2):e16316. Available from: https://mhealth.jmir.org/2020/2/e16316/
  » https://mhealth.jmir.org/2020/2/e16316/
• ²⁵ Efthymiou A, Middleton N, Charalambous A, Papastavrou E. Adapting the eHealth Literacy Scale for Carers of People With Chronic Diseases (eHeals-Carer) in a Sample of Greek and Cypriot Carers of People With Dementia: Reliability and Validation Study. J Med Internet Res [Internet]. 2019 [cited 2021 Mar 6];21(11):e12504. Available from: https://www.jmir.org/2019/11/e12504/
  » https://www.jmir.org/2019/11/e12504/
• ²⁶ Paige SR, Krieger JL, Stellefson M, Alber JM. eHealth literacy in chronic disease patients: an item response theory analysis of the eHealth literacy scale (eHEALS). Patient Educ Couns. 2017;100(2):320-6. https://doi.org/10.1016/j.pec.2016.09.008
  » https://doi.org/10.1016/j.pec.2016.09.008
• ²⁷ Han HR, Hong H, Starbird LE, Ge S, Ford AD, Renda S, Sanchez M, Stewart J. eHealth Literacy in People Living with HIV: systematic review. JMIR Public Health Surveill [Internet]. 2018 [cited 2021 Mar 6];4(3):e64. Available from: https://publichealth.jmir.org/2018/3/e64/
  » https://publichealth.jmir.org/2018/3/e64/
• ²⁸ Bailey CE, Kohler WJ, Makary C, Davis K, Sweet N, Carr M. eHealth Literacy in Otolaryngology Patients. Ann Otol Rhinol Laryngol. 2019;128(11):1013-8. https://doi.org/10.1177/0003489419856377
  » https://doi.org/10.1177/0003489419856377
• ²⁹ Guillemin F, Bombardier C, Beaton D. Cross-cultural adaptation of Health related quality of life measures: Literature review and proposed guidelines. Clin Epidemiol. 1993;46(12):1417-32. https://doi.org/10.1016/0895-4356(93)90142-n
  » https://doi.org/10.1016/0895-4356(93)90142-n
• ³⁰ Reichnheim ME, Moraes CL. Operationalizing the cross-cultural adaptation of epidemological measurement instruments. Rev Saúde Pública. 2007;41(4):665-73. https://doi.org/10.1590/S0034-89102006005000035
  » https://doi.org/10.1590/S0034-89102006005000035
• ³¹ American Educational Research Association. American Psychological Association & National Council of Measurement in Education. Standards for educational and psychological testing [Internet]. 2014. [cited 2020 Oct 05]. Available from: https://www.apa.org/science/programs/testing/standards
  » https://www.apa.org/science/programs/testing/standards
• ³² Timmerman ME, Lorenzo-Seva U. Dimensionality assessment of ordered polytomous items with parallel analysis. Psychol Methods. 2011;16(2):209-20. https://doi.org/10.1037/a0023353
  » https://doi.org/10.1037/a0023353
• ³³ Cho SJ, Li F, Bandalos D. Accuracy of the Parallel Analysis Procedure with Polychoric Correlations. Educ Psychol Meas. 2009;69(5):748-59. https://doi.org/10.1177/0013164409332229
  » https://doi.org/10.1177/0013164409332229
• ³⁴ Howard MC. A review of exploratory factor analysis decisions and overview of current practices: what we are doing and how can we improve? Int J Hum Comput Interact. 2016;32(1):51-62. https://doi.org/10.1080/10447318.2015.1087664
  » https://doi.org/10.1080/10447318.2015.1087664
• ³⁵ Briggs NE, MacCallum RC. Recovery of weak common factors by maximum likelihood and ordinary least squares estimation. Multivariate Behav Res. 2003;38(1):25-56. https://doi.org/10.1207/S15327906MBR3801_2
  » https://doi.org/10.1207/S15327906MBR3801_2
• ³⁶ Costello AB, Osborne J. Best practices in exploratory factor analysis: four recommendations for getting the most from your analysis. Practical Assessment. 2004;10(7):1-9. https://doi.org/https://doi.org/10.7275/jyj1-4868
  » https://doi.org/https://doi.org/10.7275/jyj1-4868
• ³⁷ Divitrov DM. Statistical methods for validation of assessment scale data in counseling and related fields. Alexandria, VA: American Counseling Association; 2012. 270 p.
• ³⁸ Hair JR, Black WC, Babin BJ, Anderson R, Tathm RL. Multivariate Data Analysis. 6th ed. Upper Saddle River, NJ: Pearson Prentice Hall; 2014. 928p.
• ³⁹ Ferrando PJ, Lorenzo-Seva U. Assessing the quality and appropriateness of factor solutions and factor score estimates in exploratory item factor analysis. Educ Psychol Meas. 2018;78(5):762-80. https://doi.org/10.1177/0013164417719308
  » https://doi.org/10.1177/0013164417719308
• ⁴⁰ Eaton NR, Krueger RF, Docherty AR, Sponheim SR. Toward a model-based approach to the clinical assessment of personality psychopathology. J Pers Assess. 2014;96(3):283-92. https://doi.org/10.1080/00223891.2013.830263
  » https://doi.org/10.1080/00223891.2013.830263
• ⁴¹ Pollard B, Dixon D, Dieppe P, Johnston M. Measuring the ICF components of impairment, activity limitation and participation restriction: an item analysis using classical test theory and item response theory. Health Qual Life Outcomes. 2009;7:41. https://doi.org/10.1186/1477-7525-7-41
  » https://doi.org/10.1186/1477-7525-7-41
• ⁴² Samejima F. Estimation of latent ability using a response pattern of graded scores. Psychometrika. 1969;34:-97. https://doi.org/10.1002/j.2333-8504.1968.tb00153.x
  » https://doi.org/10.1002/j.2333-8504.1968.tb00153.x
• ⁴³ Tuerlinckx F, De Boeck P. The effect of ignoring item interactions on the estimated discrimination parameters in item response theory. Psychol Methods. 2001;6(2):181-95. https://doi.org/10.1037/1082-989x.6.2.181
  » https://doi.org/10.1037/1082-989x.6.2.181
• ⁴⁴ Camilli G, Fox JP. An aggregate IRT procedure for exploratory factor analysis. J Educ Behav Stat. 2015;40(4):377-401. https://doi.org/1076998615589185
  » https://doi.org/1076998615589185
• ⁴⁵ Baker FB, Kim SH. Item response theory: parameter estimation techniques. New York: Dekker; 2004. 528 p.
• ⁴⁶ Sellbom M, Tellegen A. Factor analysis in psychological assessment research: common pitfalls and recommendations. Psychol Assess. 2019;31(12):1428-41. https://doi.org/10.1037/pas0000623
  » https://doi.org/10.1037/pas0000623
• ⁴⁷ Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika. 1951;16(3):297-334. https://doi.org/10.1007/BF02310555
  » https://doi.org/10.1007/BF02310555
• ⁴⁸ Mcdonald RP. Test theory: a unified treatment. Mahwah, NJ: Lawrence Erlbaum; 1999. 498 p.
• ⁴⁹ Dunn TJ, Baguley T, Brunsden V. From alpha to omega: a practical solution to the pervasive problem of internal consistency estimation. Br J Psychol. 2014;105(3):399-412.https://doi.org/10.1111/bjop.12046
  » https://doi.org/10.1111/bjop.12046
• ⁵⁰ Ministério da Saúde (BR). Secretaria-Executiva. Departamento de Informática do SUS. Estratégia de Saúde Digital para o Brasil 2020-2028 [internet]. Brasília: Ministério da Saúde; 2020. [cited 2021 Mar 18]. Available from: http://bvsms.saude.gov.br/bvs/ publicacoes/estrategia_saude_digital_Brasil.pdf
  » http://bvsms.saude.gov.br/bvs/ publicacoes/estrategia_saude_digital_Brasil.pdf

Edited by

• EDITOR IN CHIEF: Antonio José De Almeida Filho

• ASSOCIATE EDITOR: Hugo Fernandes

Publication Dates

• Publication in this collection
  06 Sept 2021
• Date of issue
  2022

History

• Received
  29 Dec 2020
• Accepted
  03 May 2021