Elaboration and psychometric analysis of a questionnaire to evaluate the knowledge of community health workers about breastfeeding

Abstract

Objectives:

to develop and analyze the psychometric properties of a questionnaire to assess community health workers’ knowledge on breastfeeding.

Methods:

this is a methodological study for the development of an instrument and analysis of validity and reliability. For the elaboration of items and identification of dimensions, a literature review was conducted. The items were submitted to the evaluation of a committee of judges, for apparent and content analysis. Construct validation was conducted through hypothesis test, with the participation of 282 community health workers and 19 pediatricians and obstetric nurses. For analysis and comparison of scores, the Mann-Whitney U test was used, assuming a significance level of 5%. Reliability was assessed using Cronbach's alpha coefficient and temporal consistency of the instrument was assessed using test-retest and Kappa analysis.

Results:

the final instrument presented a favorable opinion from the committee of judges. The hypothesis test showed that the questionnaire has discriminatory power to assess professionals with a higher level of knowledge (p<0.001). The Kappa test revealed that 63% of the items showed substantive to almost perfect agreement. The 32-item questionnaire showed a Cronbach's alpha of 0.794.

Conclusion:

the instrument developed was valid and reliable, allowing effective measurement of community health workers’ knowledge on breastfeeding.

Key words:
Breastfeeding; Community health workers; Family health strategy; Primary health care; Validation studies

Resumo

Objetivos:

elaborar e analisar propriedades psicométricas de um questionário para avaliação do conhecimento de agentes comunitários de saúde sobre amamentação.

Métodos:

trata-se de estudo metodológico de elaboração de instrumento e análise de validade e confiabilidade. Para a elaboração dos itens e identificação das dimensões, realizou-se revisão da literatura. Os itens foram submetidos à apreciação de comitê de juízes, para análise aparente e de conteúdo. A validação de construto foi conduzida pelo teste de hipóteses, com participação de 282 agentes comunitários de saúde e 19 pediatras e enfermeiras obstetras. Para comparação dos escores, utilizou-se o teste U de Mann-Whitney, assumindo-se um nível de significância de 5%. A confiabilidade foi aferida por meio do coeficiente alfa de Cronbach e a consistência temporal do instrumento por meio do teste-reteste e análise de Kappa.

Resultados:

o instrumento final apresentou parecer favorável do comitê de juízes. O teste de hipóteses evidenciou que o instrumento tem poder discriminatório para aferir profissionais com maior nível de conhecimento (p<0,001). O teste Kappa revelou que 63% dos itens apresentaram concordância de substantiva a quase perfeita. O questionário, com 32 itens, apresentou um alfa de Cronbach de 0,794.

Conclusão:

o instrumento apresentou-se válido e confiável que permitirá mensurar de forma eficaz o conhecimento de agentes comunitários de saúde sobre amamentação.

Palavras-chave:
Aleitamento materno; Agentes comunitários de saúde; Estratégia saúde da família; Atenção primária à; saúde; Estudos de validação

Introduction

Breastfeeding is considered the most effective way to nourish the child, from birth to the sixth month of life. The World Health Organization and the Ministry of Health in Brazil recommend exclusive breastfeeding until the age of six months.¹,² Expanding and encouraging breastfeeding might prevent 823,000 deaths each year in children younger than five years and 20,000 deaths from breast cancer.³ In addition to the nutritional benefits, breast milk protects against childhood infections, increases intelligence, and has the potential to reduce the occurrence of overweight, diabetes, and allergic disorders such as asthma and other morbidities.¹,³

The Primary Health Care (PHC) teams assume a prominent role in encouraging the practice of breastfeeding, as besides being close to families, they develop a care model that prioritizes health promotion actions.⁴ In Brazil, the PHC is operationalized by the multidisciplinary teams that make up the Family Health Strategy (FHS). All team members, in their respective areas of activity, must commit to promote breastfeeding, which can generate positive results.⁴,⁵

To be successful in promoting breastfeeding, health professionals must be properly trained and seek to incorporate, in their daily activities, the understanding of breastfeeding as a dynamic process and with different meanings for the subjects involved.⁴,⁵ In this sense, the need for the qualification of Community Health Agents (CHA) is indisputable, so that they can promote educational actions with the community, raising awareness in pregnant women and nursing mothers of the benefits and importance of breastfeeding and its effective practices.⁶,⁷ Furthermore, the actions of these professionals must be instrumentalized so that they can be developed safely and provide adequate guidance for pregnant women and nursing mothers.⁷

Permanent education programs that promote breastfeeding are essential for FHS professionals. Ideally, educational activities should be built from the identification of learning needs, which justifies the use of instruments relevant to assess the CHAs' knowledge on breastfeeding. Therefore, this study aimed to develop and analyze the psychometric properties (validity and reliability) of a questionnaire to assess CHAs’ knowledge on breastfeeding.

Methods

This is a methodological study for the elaboration and analysis of psychometric properties of a questionnaire to assess community health workers’ knowledge on breastfeeding.

The process followed four stages: 1) Identification of the literature relevant to the topic and construction of dimensions related to breastfeeding; 2) Apparent and content validation; 3) Construct validation through hypothesis testing; and 4) Reliability analysis through the analysis of internal consistency and test-retest (Figure 1).

Stage 1) Identification of the literature relevant to the topic and construction of dimensions related to breastfeeding

To identify the literature on the topic, we conducted a literature review using the descriptors "breastfeeding", "primary health care", and "community health agent". The search was performed in April 2017, based on the Virtual Health Library Portal (VHL). The inclusion criteria for this study consisted of articles in Portuguese, English, or Spanish, which dealt with the CHAs approach to breastfeeding, and were published between 2012 and 2016. Additionally, we included guidance documents for health professionals for the promotion of breastfeeding by the Ministry of Health and the World Health Organization.¹,²

The synthesis of the integrative review and the documents of official agencies allowed the elaboration of the main contents pertinent to the construction of the questionnaire and resulted in 30 items, grouped in the three dimensions: seven items for the dimension Features and care of the breasts, seven items for Introduction of new foods and breastfeeding, and 16 items for The breastfeeding process. The three dimensions considered in the study and its objectives are shown in Table 1.

The main themes within each dimension were identified and transformed into short and objective statements, which became items. Part of them was maintained as true statements, similar to the reference text and one-third of the items were randomly selected and transformed into false statements. After each statement, the response options were organized on a five-level Likert scale: "I'm sure it is right", "I think it is right", "I do not know if it is right or wrong", "I think it is wrong", “I'm sure it is wrong”.

Step 2) Apparent and content validation

The instrument developed was subjected to content and semantic structure analysis by eight specialists in the field: three obstetrical nurses, two nurses with experience in the field of breastfeeding and human milk bank, and two pediatric researchers with experience in the field of breastfeeding. The professionals in this study were appointed as a “Committee of judges”. They assessed the presence or absence of the scope and relevance criteria (content validation), objectivity, and clarity (apparent validation). Each item was rated as “very relevant”, “relevant”, “somewhat relevant”, or “irrelevant”. In the analysis of each of the judges, the ratings were scored using the Content Validity Index (CVI), in which responses with “very relevant” and “relevant” assessments should correspond to more than 80% of the total respondents so that the item would remain in the questionnaire. After analysis by all members of the committee of judges, the instrument was reformulated in its writing, according to the guidelines and suggestions received and the 30 items were maintained, as each of them was classified as relevant or very relevant for more than 80% (at least seven of the eight members) of the committee of judges. Regarding the apparent validation, all items were also considered clear and objective. At the suggestion of the committee of judges and considering the relevance, two more items that addressed important aspects about the CHA’s knowledge were added to the instrument.

Figure 1
Stages of elaboration and validation of the instrument “Community health agents’ knowledge on breastfeeding”.

Table 1
Objectives of the dimensions of the instrument "Community health workers' knowledge on breastfeeding".

Dimension	Objectives
Characteristics and care of the breasts	Evaluate the knowledge of the CHA on the characteristics and care of the breasts during the prenatal period and lactation.
Introduction of new foods and breastfeeding	Assess the CHAs' knowledge on the weaning process and on the need and general aspects of the introduction of new foods during the first months of the child's life.
Breastfeeding process	Assess the CHA's knowledge on the breastfeeding process, benefits, and care.

• CHA = Community Health Agents .

Step 3) Construct Validation

The preliminary version of the instrument “Knowledge of community health agents on breastfeeding” was applied to CHAs registered in FHS teams in three municipalities in the north of Minas Gerais (Montes Claros, Taiobeiras, and Pirapora). In addition, they were applied to pediatric doctors and obstetric nurses, who were considered more knowledgeable or “specialists” in breastfeeding, under the terms of the present study.

The instrument applied to the CHAs included some items of characterization of the respondents, such as sex, age, and length of experience in the FHS. Regarding the knowledge assessment questionnaire, professionals were asked to record their level of knowledge on the topic, indicating the most pertinent option, according to the Likert scale previously presented.

We considered correct answers those that showed partial or full agreement with the true statements, as well as partial or full disagreement with the false statements. All answers from the CHAs and the “specialists” were transformed into scores, by adding the values assigned on the Likert scale to the items that were part of the instrument, assigning a point for each correct answer.

We also conducted the hypothesis test, which was proposed to verify whether the instrument was able to discriminate between pediatricians and nurses and CHAs, who should achieve higher and lower scores, respectively. Furthermore, we compared scores between the CHAs, according to their period of experience in the FHS teams. The comparison of the item scores between the groups was performed using the Mann-Whitney U test, considering that the values of the scores did not present a normal distri-bution. The level of significance assumed was 5% (p<0.05).

Step 4) Reliability analysis

The reliability analysis was conducted through the calculation of Cronbach's alpha, the item/total correlation from the correlation matrix of all items of the instrument, and the analysis of the Kappa statistics, with the performance of the test-retest.

The test-retest evaluated the temporal stability of the questionnaire applied to 10% of the respondents, in an interval of one to two weeks, as recommended in studies of this nature.⁸ The classification for the Kappa concordance test between the two applications considered the following parameters: absence of agreement <0, poor agreement 0-0.19, slight agreement 0.20-0.39, moderate agreement 0.40-0.59, substantive agreement 0.60-0.79, almost perfect agreement 0.80-1.00.⁹

All data were entered and analyzed using the IBM statistical Software SPSS version 22. The study was conducted following the ethical guidelines for research involving human beings. Participation was voluntary, with the signing of a free and informed consent term, and the study project was approved by the Research Ethics Committee of Faculdades Integradas Pitágoras, under opinion # 2,521,430/2018.

Results

Regarding the elaboration of the items, the synthesis of the themes identified in the literature resulted in 30 statements. After analysis by the specialists, two more items were added, which were randomly arranged on the instrument. All items achieved satisfactory evaluation according to the CVI, that is, at least seven of the eight judges consider the item as "relevant" or "very relevant", according to the proposed criteria. The final semantic structure of each item was unanimously approved by the judges. In the construct validation stage, the instrument was applied to 282 CHAs registered in FHS teams and 19 pediatricians and obstetric nurses. Among the CHAs who participated in the validation process, there was a predominance of females (92.2%), aged 31 or over (70.2%). Most of these professionals reported working at the FHS for more than five years (53.2%). Approximately two thirds of the professionals answered that they had already participated in some training or course on breastfeeding (62.5%). In relation to the three dimensions, there was a higher average of correct answers for Characteristics and breast care (79.3%), followed by the evaluation on The breastfeeding process (74.9%) and a lower percentage of correct answers for Introduction of new foods and breastfeeding (61.6%).

Among pediatricians and obstetric nurses (“specialists”), the percentage of correct answers for each item varied from 44.4% to 100%. Lower percentages of correct answers were registered for the item "If the mother needs to be away from the child for a few hours, the ideal option is to remove the milk and leave it in a cool place to be offered during her absence".

Regarding the final scores of the CHAs, the percentage of correct answers for each item ranged from 19.9% to 98.6%. Lower percentages of correct answers were registered for the items “It is necessary to alternate breasts after some time of breastfeeding so that the baby breastfeeds from both breasts”, “The baby must be breastfed regularly: for example, every 2 hours in the first week, and every 3 hours thereafter”, and “If there is a decrease in milk production, and the baby appears to be hungry, complementation should be started immediately”.

Regarding the hypothesis test, the comparison of scores between CHAs and specialists identified an important difference between the two groups, with better performance by specialists (p<0.001). Moreover, comparisons were made between the CHAs, considering sex and period of experience in the FHS. There was a significant difference between the CHAs’ scores with experience of more than five years (p=0.019), but there was no difference between sexes (p=0.808) (Table 2).

After the instrument's reproducibility test, we observed that only two items (Q04 and Q20) presented an agreement index considered “poor”. Ten items (Q03, Q07, Q09, Q10, Q11, Q13, Q21, Q24, Q26, and Q30) showed moderate agreement, and all other items of the instrument showed agreement above 60%, which represents substantive to almost perfect agreement.⁹

The 32-item questionnaire showed a Cronbach's alpha of 0.794. The item-total correlation from the correlation matrix of all items in the instrument showed that some had a low correlation (<0.3), but their removal would only slightly alter the Cronbach's alpha value. The final instrument is shown in Table 1.

Discussion

The present study enabled the development of a valid and reliable instrument for assessing the CHAs’ knowledge on breastfeeding. The product is important to guide the implementation of educational activities to improve the practice of CHAs in relation to the promotion of breastfeeding, identifying, a priori, myths and misconceptions that should be better addressed. The elaborated instrument also has the merit of arousing interest of CHAs on the topic, which can be pointed out as a positive point and that stimulates the approach to the subject. In addition, this instrument is simple, objective, relatively short, and easy to understand.

Table 2
Comparative analysis between the average score of correct answers for the questionnaire to assess knowledge on breastfeeding. Montes Claros, MG, Brazil, 2019.

Variables	̅X ± SD	p*
Professional category
Community Health Agents	24.1 ± 3.9	0.001
Pediatricians / Obstetric nurses	29.2 ± 1.6
Time of action of the CHA in the FHS (years)
< 5	23.4 ± 4.4	0.019
>5	24.7 ± 3.4
Sex
Male	24.0 ± 4.1	0.808
Female	24.1 ± 3.9

• ^*
  Mann-Whitney U test; CHA = Community Health Agent; FHS = Family Health Strategy.

Considering that there are no instruments to verify CHAs’ knowledge on breastfeeding, our findings could not be discussed by comparing them with similar scales. The national literature contains some studies that assess the knowledge of CHAs on the topic.⁷,¹⁰,¹¹ However, these studies do not use validated or reliable instruments. The use of instruments with good psychometric properties increases quality in studies and plays an important role in research, clinical practice, and health assessment.⁸

All aspects recommended for the construction of evaluative instruments in the health area were followed, starting with the Elaboration of items, from the literature.¹² Regarding Content Validation, the semantic analysis was performed by a more sophisticated sample (of greater ability) of the target population, verifying that the items of the instrument are clear and intelligible for the group to be evaluated. Thus, the instrument was submitted to the committee of judges on the subject and presented an acceptable agreement index (that is, a content validity index greater than 0.80).⁸ As per suggestions of the committee of judges, two questions were added which addressed “Breast milk when expressed and stored must be offered in cups or small glasses” and “Women with large milk production can be referred to the Human Milk Bank to be donors”.

Although not identified as essential aspects in the initial assessment of the authors, the questions proved to be unanimous among the judges and, therefore, were incorporated into the instrument. The added items portray those breastfeeding women who produce a volume of milk beyond the baby's need as potential donors in a human milk bank. For this, she must be healthy and not use drugs that prevent donation. Human milk banks play a social role and are generally linked to maternity hospitals. The donated milk is analyzed and subsequently distributed, under the doctor's or nutritionist's prescription, to children with low weight and/or premature infants who cannot be fed directly on the mother's breast.¹²,¹³ The other item inserted at the suggestion of the judges highlights that the expressed milk must be offered in a small glass, being an important aspect in combating the use of bottles, which can promote early weaning.¹⁴,¹⁵ This is important information that should be known by the CHAs, as the Ministry of Health clarifies that women who are breastfeeding, in case of doubts about how to conserve or offer breast milk to the child, should seek support at the Basic Health Unit where she is being assisted.

In the Construct Validation, the hypothesis test was able to properly discriminate individuals who have greater knowledge, which was observed in the comparison of the professional categories and in relation to the period in which CHA worked in the FHS. A study by Moimaz et al.⁷ demonstrated that CHAs had limited knowledge on the practice and promotion of breastfeeding and that the monitoring of women who are breastfeeding is compromised by the lack of training courses for these professionals.

In relation to the measured scores, a percentage of 44% of correct answers drew attention to one of the members of the team of specialists (pediatricians and obstetric nurses). The detailed analysis revealed that only one professional in this group had a low score. However, despite the high scores for most items, this result, coupled with the fact that some items did not reach 100% of correct answers among specialists, points out that the group is not necessarily uniform and there is a need for regular training courses/update for these professionals, who should be a reference in relation to breastfeeding management.

In the Reliability Analysis, the 32-item questionnaire was analyzed in a unidimensional manner and presented a satisfactory level of internal consistency and good reliability and reproducibility, as shown in the Kappa test statistics. We decided to keep the two items that presented poor agreement, since they dealt with cases of breast engorgement (“cobbled milk”), highlighting that it is important to massage the breast before breastfeeding and to make cold compresses on the site. The item addresses a relatively frequent issue, for which CHAs’ action is very important. The literature is controversial on the topic.¹⁶ However, cold compresses do not cause harm when applied to the engorged breast and generate temporary vasoconstriction that decreases blood flow and edema, thus being considered effective to relieve pain.¹⁶,¹⁷

The other item with low agreement highlighted that "Breast milk is a complete food and meets all the needs of the child's body until the 6th month and protects against infections". It is also a fundamental aspect of educational practice for CHAs. Several records in the literature emphasize the nutritional and immunological values of breast milk.¹⁸-²¹ For all its characteristics, breast milk is recommended as the ideal food,¹,² being unique and unmatched, and totally adapted to the needs of the child throughout the first months of life and there is no other milk that dynamically contains antibodies and that protects the child, during breastfeeding, from common infections.²²

Despite the relevance of the developed instrument, it is necessary to consider some limitations. The sample of respondent professionals was allocated according to the ease of access to the municipalities. Although there was no intentionality, as the CHAs’ identity was unknown, this allocation for convenience may represent a bias. Another aspect is that there was no factorial analysis, which has been recently considered as very relevant for the psychometric analysis of health assessment instruments. However, factor analysis is a structural analysis of the instrument and does not represent the only form of construct validation and the hypothesis test performed here is a good alternative for instruments that assess knowledge. Additionally, the minimum number of respondents recommended for the factor analysis is ten for each item,²³ which was not possible in the present study. Despite these limitations, the lack of previous scales or questionnaires aimed at the studied public highlights the importance of the elaborated product.

The instrument to assess the CHAs' knowledge on breastfeeding proved to be valid and reliable. Furthermore, it will allow to effectively measure the knowledge of this professional group and it can be considered as a tool that will contribute or direct educational actions for this audience. In addition, it can be useful to guide actions in public health and public policies for guidance and promotion of breastfeeding, as better-qualified professionals will offer assistance with better results. The instrument is available for use in studies, and further analyses may be useful to improve it.

Referências

• ¹
  WHO (World Health Organization). Guideline: protecting, promoting and supporting breastfeeding in facilities providing maternity and newborn services. Geneva; 2017. Disponível em: https://www.who.int/publications/i/item/9789241550086
  » https://www.who.int/publications/i/item/9789241550086
• ²
  Brasil. Ministério da Saúde. Estratégia Nacional para Promoção do Aleitamento Materno e Alimentação Complementar Saudável no Sistema Único de Saúde: manual de implementação. 1 ed. Brasília, DF; 2015.
• ³
  Victora CG, Bahl R, Barros AJ, França GV, Horton S, Krasevec J, et al. Breastfeeding in the 21st century: epidemiology, mechanisms, and lifelong effect. Lancet. 2016; 387 (10017): 475-90.
• ⁴
  Melo LCO, Nakano AMS, Monteiro JCS, Furtado MCC. Atributos da atenção primária à saúde na atenção ao aleitamento materno. Texto Contexto Enferm. 2019; 28: e20170516.
• ⁵
  Alves ALN, Oliveira MIC, Moraes JR. Iniciativa Unidade Básica Amiga da Amamentação e sua relação com o aleitamento materno exclusivo. Rev saúde pública. 2013; 47 (6): 1330-40.
• ⁶
  Gavine A, MacGillivray S, Renfrew MJ, Siebelt L, Haggi H, McFadden A. Education and training of healthcare staff in the knowledge, attitudes and skills needed to work effectively with breastfeeding women: a systematic review. Int Breastfeed J. 2017; 12:6.
• ⁷
  Moimaz SAS, Serrano MN, Garbin CAS, Vanzo KLT, Saliba O. Agentes comunitários de saúde e o aleitamento materno: desafios relacionados ao conhecimento e à prática. Rev CEFAC. 2017; 19 (2): 198-212.
• ⁸
  Souza AC, Alexandre NMC, Guirardello EB. Propriedades psicométricas na avaliação de instrumentos: avaliação da confiabilidade e da validade. Epidemiol Serv Saúde. 2017; 26 (3): 649-59.
• ⁹
  Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977; 33 (1): 159-74.
• ¹⁰
  Teixeira MA, Araujo VM, Ribeiro VM, Luz RT. Vivências de agentes comunitários de saúde na prática cuidativa em aleitamento materno. Rev Bras Promoç Saúde. 2016; 29 (Supl): 93-101.
• ¹¹
  Paschoal T, Tamayo A. Construção e validação da Escala de bem-estar no trabalho. Aval Psicol. 2008; 7 (1): 11-22.
• ¹²
  Fonseca RMS, Milagres LC, Franceschini SCC, Henriques BD. O papel do banco de leite humano na promoção da saúde materno-infantil: uma revisão sistemática. Ciên Saúde Coletiva. 2021; 26 (1) 309-18.
• ¹³
  Figueiredo MCD, Bueno MP, Ribeiro CC, Lima PA, Silva IT. Banco de leite humano: o apoio à amamentação e a duração do aleitamento materno exclusivo. Rev Bras Crescimento Desenvolv Hum 2015; 25 (2): 204-10.
• ¹⁴
  Lang S, Lawrence CJ, Orme RL. Cup feeding: an alternative method of infant feeding. Arch Dis Child. 1994; 71 (4): 365-9.
• ¹⁵
  Penny F, Judge M, Brownell E, McGrath JM. Cup Feeding as a Supplemental, Alternative Feeding Method for Preterm Breastfed Infants: An Integrative Review. Matern Child Health J. 2018; 22 (11): 1568-79.
• ¹⁶
  Sousa L, Haddad ML, Nakano AMS, Gomes FA. Terapêutica não farmacológica para alívio do ingurgitamento mamário durante a lactação: revisão integrativa da literatura. Rev Enferm USP. 2012; 46 (2): 472-9.
• ¹⁷
  Mangesi L, Dowswell T. Treatments for engorgement during lactation. Cochrane Database Syst Rev. 2010; 9: CD006946.
• ¹⁸
  Palmeira P, Carneiro-Sampaio M. Immunology of breastmilk. Rev Assoc Med Bras. 2016; 62 (6): 584-93.
• ¹⁹
  Blewett HJH, Cicalo MC, Holland CD, Field CJ. The immunological components of human milk. Adv Food Nutr Res. 2008; 54: 45-80.
• ²⁰
  Gertosio C, Meazza C, Pagani S, Bozzola M. Breastfeeding and its gamut of benefits. Minerva Pediatr. 2016; 68 (3): 201-12.
• ²¹
  Silva GA, Costa KA, Giugliani ER. Infant feeding: beyond the nutritional aspects. J Pediatr (Rio J). 2016; 92 (Suppl 1): S2-7.
• ²²
  Samuel TM, Zhou Q, Giuffrida F, Munblit D, Verhasselt V, Thakkar SK. Nutritional and Non-nutritional Composition of Human Milk Is Modulated by Maternal, Infant, and Methodological Factors. Front Nutr. 2020; 7: 576133.
• ²³
  Hair JF, Black WC, Babin BJ, Anderson RE, Tatham RL. Análise multivariada de dados. 6 ed. São Paulo: Bookman; 2009.

Publication Dates

• Publication in this collection
  10 Sept 2021
• Date of issue
  Apr-Jun 2021

History

• Received
  28 Feb 2020
• Accepted
  26 Mar 2021
• Preprint posted on
  4 Jan 2021