Validity and reproducibility of a Food Frequency Questionnaire for German descendants living in Brazil

Validação e reprodutibilidade de um Questionário de Frequência Alimentar para descendentes Alemães residentes no Brasil

ABSTRACT

Objective

This study evaluated reproducibility, relative validity, using a 24-hour recall questionnaire as a reference standard, and estimated calibration factors for a food frequency questionnaire adapted for use with German descendants living in Brazil.

Methods

The target population consisted of 50 volunteers, of both genders, aged over 20 years, living in a German colonization city in southern Brazil. The food frequency questionnaire was applied twice, in the first and third months of the investigation. During this period, three 24-hour recalls were applied, with an interval of one month between them. Reproducibility was estimated by the intraclass correlation coefficient. Validity was tested by the intraclass correlation coefficient, weighted kappa test and Bland-Altman method. Calibration factors were estimated using linear regression.

Results

Among the food frequency questionnaires, there was a strong correlation for energy and most of the nutrients corrected for energy. There was a weak correlation between a food frequency questionnaire and a 24-hour dietary recall. However, the exact concordance in the categorization in tertiles among the instruments ranged from 28% (vitamin A) to 52% (fiber and potassium). Gross values of the food frequency questionnaire were reduced with the calibration and approached the consumption data estimated by the 24-hour dietary recall.

Conclusions

The food frequency questionnaire showed good reproducibility, however, weak correlation with the 24-hour dietary recall. The calibration of the data obtained by the food frequency questionnaire brought them closer to the reference method.

Keywords
Food Consumption; Surveys and Questionnaires; Tests Reproducibility; Validation Studies

RESUMO

Objetivo

Avaliar a reprodutibilidade, validade relativa, usando questionário de recordatório de 24 horas como padrão de referência, e estimar fatores de calibração de um questionário de frequência alimentar adaptado para uso com descendentes alemães vivendo no Brasil.

Métodos

A população alvo foi constituída de 50 voluntários, de ambos os sexos, com idade superior a 20 anos, residentes em uma cidade de colonização alemã do Sul do Brasil. O questionário de frequência alimentar foi aplicado duas vezes, no primeiro e terceiro mês. Neste período, foram aplicados três recordatórios de 24 horas, com intervalo de um mês entre eles. A Reprodutibilidade foi estimada pelo coeficiente de correlação intraclasse. Validade foi testada pelo coeficiente de correlação intraclasse, teste kappa ponderado e método Bland-Altman. Fatores de calibração foram estimados por meio de regressão linear.

Resultados

Entre os questionários de frequência alimentar, houve forte correlação para energia e a maioria dos nutrientes corrigidos para energia. Verificou-se fraca correlação entre questionário de frequência alimentar e recordatório de 24 horas. Porém, a concordância exata na categorização em tercis entre os instrumentos variou de 28% (vitamina A) a 52% (fibra e potássio). Valores brutos do questionário de frequência alimentar foram reduzidoscom a calibração e se aproximaram dos dados de consumo estimados pelo recordatório de 24 horas.

Conclusão

O questionário de frequência alimentar demonstrou boa reprodutibilidade, porém, fraca correlação com o recordatório de 24 horas. A calibração dos dados obtidos pelo questionário de frequência alimentar aproximou-os do método de referência.

Palavras-chave
Consumo de Alimentos; Inquéritos e Questionários; Reprodutibilidade dos Testes; Estudos de Validação

INTRODUCTION

The eating habits of a nation do not stem only from the survival instinct and man’s eating need. They are representations of their history, geography, climate, social organization and have deep roots in the social identity of individuals. They are rooted in their lifestyle and collective practices, depending on the prevailing culture. Furthermore, eating habits are influenced by socioeconomic factors [¹].

In the late nineteenth century, several groups, largely formed by Prussians (from Hunsrück, Eifel, Rhineland in general, Pomerania, Westphalia and Silesia) and their descendants landed in Brazil and formed colonies in the Northeast, Southeast and South regions of the country [²]. According to research carried out based on historical sources to classify the ancestry of Brazilian surnames, it was shown that, in a universe of 46,801,772 Brazilian names reviewed, 1,525,890 or 3.3% of individuals had the only or last name indicating a German ancestry [³].

Eating habits are a direct reflection of the behavior patterns of a given culture, as they are a reference of the cultural customs and traditions of a people. Similar to other peoples, the German immigrant brought to Brazil gastronomic traits and eating habits that were incorporated not only by German descendants, but also influenced the formation of the Brazilian food culture [⁴].

Whereas (a) The assessment of the quality of the populations diet is essential for the learning of dietary patterns and the articulation of causal relationships between diet and diseases [¹]; (b) The challenges of researchers in the use of food surveys due to the complexity of analyzing qualitatively and quantitatively human intake [⁵]; and (c) The expressive number of Brazilians with German ancestry that inhabit many regions of Brazil, highlights the importance of validating a quantitative Food Frequency Questionnaire (FFQ), which includes food from German cuisine, developed for the Brazilian population that has cultural ties consistent with the Germanic tradition.

Dietary assessment methods, including food frequency questionnaires, are useful and valid to assess diet-disease relationships in epidemiological studies and have contributed to science for decades through studies investigating the association between dietary intake and the occurrence of clinical outcomes, in general related to chronic non-communicable diseases [⁵,⁶]. It is a culturally specific instrument; this is why it is important to assess whether each new FFQ presents reproducible and valid results for the target population of the study [⁷].

Given the above, the objective was to evaluate the reproducibility and validity of the FFQ adapted for populations of German descendants residing in Brazil and to estimate the calibration factors.

METHODS

To compose the sample to be assessed, the first volunteers of both genders, aged twenty years or more, who had been living in the municipality of Pomerode for at least six months, who were interviewed in the first stage of the longitudinal population-based study, were selected. The study was entitled “Study of the Living Conditions and Health of the Population of Pomerode, SC”. Pomerode is a municipality in Santa Catarina that seeks to legitimize the slogan of the “most German city in Brazil” and that has a strong presence of German eating habits among its descendants of German immigrants. The following inclusion criteria were adopted: participating in the aforementioned population study and accepting to participate in all stages of this survey. Individuals with physical and mental limitations were excluded. A minimum adequate sample size was complied with for this type of study [⁷]. This investigation was approved by the Research Ethics Committee of the Universidade Regional de Blumenau, under opinion number 30952514.1.0000.5370.

The FFQ was applied in two stages: in the first contact with the participant (FFQ1) (first month) and at the end of this study (third month, in sequence) (FFQ2). Three (03) 24-Hour Dietary Recall (24hDR) were applied; the 24hDR1 together with FFQ1 (first month), the 24hDR2 (second month), and the 24hDR3 at the same time as FFQ2 (third month). The time interval between the applications of the first and third 24hDR and the FFQ1 and FFQ2 was 3 months. Validation studies recommend that the process, in general, take place over a period of six months to one year. In this study, gender and age were considered for sample identification.

FFQ1 and 24hDR1 were applied at the university by two trained interviewers. The applications of 24hDR2, FFQ2 and 24hDR3 were carried out by the same interviewers at the city General Polyclinic, in a similar setting. The interviewers were qualified in accordance with Standard Operating Procedures (SOP) developed for this study. In one of the SOPs there was a detailed description of how to ask questions the frequency and number of servings consumed of the foods presented in the FFQ, and in another, a detailed description of how to inquire about times, types of food and the amount eaten the day before (24hDR).

The FFQ evaluated was adapted from an already validated FFQ (ELSA-Brasil) [⁸]. The adaptation occurred through communities focus groups in the city of Pomerode, SC [⁹]. In the FFQ adaptation process, the following foods were included: bread fried in lard, bread with sardines and egg, apfelstrudel, cat ear, duck or stuffed duck, sweet potato gnocchi, meat dumpling, white black pudding (leberwurst) and dark (blutwurst), butter biscuit, homemade mayonnaise, colonial cheese, kochkäse, pork fat, colonial milk, bovine tongue, cuka cake, pork head jelly (sülse), cream, duck blood soup (schwarzsauer), loquat (plum) and draft beer. Excluded foods were: cajá, winged yam, yam, cooked plantain, cashew, small crab, crab, acarajé, Bahia food (vatapá, caruru and fish moqueca), chicory, okra, pistachio, cottage, buffalo mozzarella, polengui cheese and flatbread. The FFQ object of this survey is an instrument to assess quantitative food consumption, over the last 12 months, containing 116 food items, an option to fill in the quantity (number of servings consumed according to the household measure previously defined in the questionnaire) and nine consumption options (more than three times a day, two to three times a day, once a day, five to six times a week, two to four times a week, once a week, once to three times a month, never or almost never and seasonal consumption). When the respondent reported not consuming a certain food or consuming it very few times in the last year, the frequency of consumption was indicated as never or almost never; however, when the respondent who had a vegetable garden or consumed the food only at the time of its manufacture/harvest the frequency of consumption became seasonal. To assist the volunteers in their answers, a didactic material was produced with information regarding the frequency of consumption included in the FFQ and a photographic manual with the foods included in the FFQ list and its home measure.

For the validation of the FFQ, 24hDR was used as a reference method. Volunteers were encouraged to report in detail what they ingested in terms of food and beverages in the last 24 hours before the interview. In addition, they informed the size or volume of the portion ingested, with the help of a set of dishes. The first and the last 24hDR (24hDR1 and 24hDR3) were applied on a random day of the week, whereas the 24hDR2 was applied on Monday, to cover the food consumption on the weekend (Sunday). To quantify the nutrients from the FFQ and 24Rh food items, the USDA National Nutrient Database for Standard Reference – Release 27, from the United States Department of Agriculture, was used, as it includes a complete database (nutrients and foods).

The estimated consumption of the FFQ was based on the result of the multiplication of the frequency of consumption times the number of servings times the centesimal composition of the nutrient. To calculate the food consumed in 24hDR1, the nutritional composition of the portion of the food/preparation consumed was used. The nutritional composition and the homemade measures of the regional preparations were obtained through technical cards. The preparation of the technical card was carried out by one of the researchers (nutritionist) with the help of local residents and considered the ingredients, the way of preparing the recipe, the yield (to define the portions) and food group of the recipe. The quantification of nutrients from these preparations was performed using the USDA table.

The dietary variables studied were: total energy (kcal), carbohydrates (g), proteins (g), lipids (g), fiber (mg), calcium (mg), iron (mg), potassium (mg), selenium (mcg), zinc (mg), vitamin A (UI), vitamin C (mg) and vitamin E (mg). These variables were chosen because they are of interest to the population-based study, entitled “Study of the Living and Health Conditions of the Population of Pomerode, SC”.

Energy and nutrient intake were described using means and Standard Deviations (SD) for the values of FFQ1, FFQ2 and mean of the three 24hDR. Data normality was investigated using the Shapiro-Wilk test. Due to their non-normal distribution, energy and nutrients were transformed into their natural logarithm. Subsequently, the means were de-attenuated considering the intra- and inter-individual variance ratios. The nutrients of FFQ1 and FFQ2 and the average of 24hDR were corrected for energy, using the residual method [¹⁰]. In the linear regression analysis, energy intake was considered as an independent variable and nutrient intake as a dependent variable.

Reproducibility between FFQ1 and FFQ2 and validity between FFQ2 and the mean of the three 24hDRs were estimated by calculating the Intraclass Correlation Coefficient (ICC) and their respective 95% Confidence Intervals (95%CI). The agreement between the FFQ2 and the reference method (mean of the three days of 24hDR, log transformed nutrients) was measured by the weighted kappa test and by the test proposed by Bland & Altman [¹¹]. To calculate the kappa and 95%CI values, energy and nutrient intake was divided into tertiles, which allowed classifying the type of agreement into exact, adjacent and disagreement. The Bland-Altman method allowed us to estimate the limits of agreement (LoA) and their 95% CI, which identifies in which range the values of the differences between the two methods are expected to be found. In addition, FFQ2 intake data were calibrated using the 24hDR method, through linear regression [¹²]. The mean nutrient intake of the three days of 24hDR, adjusted for energy, was considered as the dependent variable and the FFQ2 as the independent variable. The linear regression constant (α) and the slope of the curve (λ) were the parameters considered in the error calibration, according to the formula described below: FFQ2 calibrated = α+λFFQ2. The analyses were performed using the Stata version 12 application and the significance level adopted was p<0.05.

RESULTS

Out of the 120 volunteers interviewed in the first months of the population-based study, 50 agreed to participate in all stages of this work. The total sample consisted of 50 individuals, 30 (60%) women and 20 (40%) men, with a mean age of 54.1±15.1 years, and 14 (28%) were in the age group of 21 to 49 years and 36 (72%) between 50 to 77 years. The energy and nutrients mean consumption obtained by the two FFQs and the mean of the 24hDR are shown in Table 1. The values of energy and all nutrients intake were overestimated by both FFQs compared to the mean of the 24hDR.

Table 1
Means and standard deviations of energy and nutrients assessed using the Food Frequency Questionnaire 1, Food Frequency Questionnaire 2 and the average of the three 24 hours dietary recalls.

Energy and Nutrients	FFQ 1*			FFQ 2*			24hDR*
	Average	SD		Average	SD		Average	SD
Energy (Kcal)	2,492.3	845.9		2,534.9	815.0		1,989.9	573.5
Protein (g)	107.9	41.8		106.0	42.7		91.1	30.9
Lipid (g)	149.8	143.3		186.0	449.8		95.5	52.0
Carbohydrate (g)	359.9	125.0		363.6	112.5		280.4	80.3
Fiber (g)	35.7	13.7		35.4	13.3		22.3	8.7
Calcium (mg)	1,156.6	426.0		1,119.4	407.4		849.7	311.1
Iron (mg)	17.8	6.9		18.0	6.9		14.5	5.2
Potassium (mg)	4,486.7	1.623.2		4,373.7	1.370.2		3,315.3	1.183.5
Zinc (mg)	93.8	104.5		83.8	124.1		33.9	45.2
Vitamin C (mg)	279.7	194.9		330.9	263.9		117.8	120.3
Vitamin A (IU)	14,910.1	6.487.1		13,609.1	6.803.3		7,470.4	5.227.6
Vitamin E (mg)	9.9	5.5		9.1	3.6		6.2	6.4
Selenium (mg)	147.9	66.2		148.2	61.4		123.6	42.0

• Note:
• ^*
  Gross nutrients; 24hDR: 24 Hours Dietary Recalls; FFQ1: Food Frequency Questionnaire 1; FFQ2: Food Frequency Questionnaire 2; SD: Standard Deviation.

Table 2 shows the correlation coefficients obtained between FFQs 1 and 2. After correction for energy, there was an increase in the nutrients ICCs, except for zinc and vitamin C. A strong correlation was observed between FFQ1 and FFQ2 for energy and the nutrients: protein, carbohydrates, lipid, fiber, calcium, iron, potassium and selenium (corrected for energy).

Table 2
Intraclass correlation coefficient and confidence intervals between Food Frequency Questionnaire 1 and Food Frequency Questionnaire 2.

Energy and Nutriengs	ICC*	95%CI	ICC**	95%CI
Energy (Kcal)	0.70	0.56-0.84	-	-
Protein (g)	0.58	0.40-0.77	0.70	0.56-0.84
Lipid (g)	0.45	0.36-0.54	0.64	0.50-0.78
Carbohydrate (g)	0.69	0.55-0.84	0.70	0.56-0.84
Fiber (g)	0.73	0.59-0.86	0.70	0.55-0.84
Calcium (mg)	0.59	0.41-0.77	0.70	0.55-0.84
Iron (mg)	0.67	0.51-0.83	0.70	0.56-0.85
Potassium (mg)	0.67	0.52-0.82	0.69	0.55-0.84
Zinc (mg)	0.60	0.43-0.78	0.40	0.26-0.53
Vitamin C (mg)	0.59	0.42-0.76	0.36	0.23-0.49
Vitamin A (IU)	0.43	0.21-0.66	0.60	0.44-0.76
Vitamin E (mg)	0.35	0.13-0.57	0.59	0.45-0.73
Selenium (mg)	0.61	0.43-0.79	0.70	0.57-0.85

• Note:
• ^*
  Analysis performed on log-transformed energy and nutrients;
• ^**
  Analysis performed on log-transformed nutrients and adjusted for energy. CI: Confidence Intervals; ICC: Intraclass Correlation Coefficient.

Table 3 demonstrates the validation tests of the FFQ2 in relation to the average of 24hDR. Higher and lower ICC values not adjusted for energy were found for zinc (0.39) and lipid (0.00), respectively. After adjusting for energy, it was found that the ICCs had lower values for vitamin C (0.00) and higher values for protein (0.30). Exact agreement in tertile categorization for nutrient de-attenuated estimates ranged from 28% (vitamin A) to 52% (fiber and potassium). On the other hand, disagreement in the categories (opposite tertiles) ranged between 10% (zinc) and 16% (fiber, potassium and vitamin E). The weighted kappa value ranged from 0.05 (vitamin A) to 0.28 (protein, fiber, iron, potassium, zinc and vitamin C).

Table 3
Intraclass correlation coefficient, confidence intervals, percentage of agreement, Weighted Kappa statistic and 95% confidence interval between food frequency questionnaire 2 and the average of the three 24 hours dietary recall.

Energy and Nutrients	ICC*	95%CI	ICC**	95%CI	Agreement*		Weighted Kappa*	95%CI
					E%	O%
Energy (Kcal)	0.27	0.08-0.46	--	--	40.0*	12.0*	0.19	0.06-0.36
Protein (g)	0.25	0.02-0.49	0.30	0.10-0.51	50.0	14.0	0.28	0.08-0.31
Lipid (g)	0.00	-0.06-0.06	0.22	0.06-0.38	32.0	12.0	0.10	0.08-0.28
Carbohydrate (g)	0.24	0.05-0.43	0.23	0.06-0.40	44.0	12.0	0.23	0.21-0.36
Fiber (g)	0.26	0.12-0.41	0.14	0.03-0.24	52.0	16.0	0.28	0.13-0.44
Calcium (mg)	0.31	0.11-0.50	0.22	0.06-0.40	42.0	14.0	0.19	0.14-0.24
Iron (mg)	0.33	0.12-0.54	0.27	0.08-0.47	48.0	12.0	0.28	0.25-0.47
Potassium (mg)	0.23	0.03-0.43	0.22	0.06-0.39	52.0	16.0	0.28	0.15-0.38
Zinc (mg)	0.39	0.26-0.51	0.10	0.02-0.18	46.0	10.0	0.28	0.24-0.39
Vitamin C (mg)	0.16	0.02-0.29	0.00	0.00-0.00	48.0	12.0	0.28	0.10-0.32
Vitamin A (IU)	0.14	-0.04-0.31	0.08	0.02-0.15	28.0	12.0	0.05	-0.02-0.12
Vitamin E (mg)	0.05	-0.16-0.25	0.08	0.02-0.15	44.0	16.0	0.19	0.14-0.41
Selenium (mg)	0.17	-0.06-0.40	0.27	0.08-0.45	40.0	12.0	0.19	0.11-0.41

• Note:
• ^*
  Analysis with energy and nutrients log- transformed and 24hDR de-attenuated;
• ^**
  Analysis performed on log-transformed nutrients, adjusted for energy and 24hDR reduced. CI: Confidence Intervals; ICC: Intraclass Correlation Coefficient.

Figure 1 shows the graph of the Bland-Altman analysis, which studies the agreement of protein and iron nutrients measured by FFQ2 and 24hDR. The agreement limits varied between negative and positive values. It was observed that nutritional intake was overestimated by the FFQ compared to the 24hDR. This characteristic was also observed for the other nutrients.

Figure 1
Analysis of dispersion of iron and protein nutrients between food frequency questionnaire 2 and the average of the three 24 hours dietary recalls.

Table 4 presents the λ calibration factor, which ranged from 0.00 (lipid) to 0.32 (fiber). The calibrated values of the FFQ2 were reduced in relation to the raw values and approached the consumption data estimated by the 24hDR.

Table 4
Calibration parameters (α and λ), confidence intervals, mean intake and standard deviation of energy and nutrients in the food frequency questionnaire 2, adjusted and calibrated.

Energy and Nutrients	α*	95%CI	λ*	95% CI	FFQ2** (SD)	FFQ2 **** (SD)
Energy (Kcal)	1327.05	822.02-1832.08	0.26	0.07-0.45	2534.92 (814.97)***	1992.48(213.94)
Protein (g)	69.34	46.30-92.38	0.21	0.00-0.41	104.16(35.24)	91.05(7.35)
Lipid (g)	96.44	80.05-112.82	0.00	-0.03-0.03	122.92(39.47)	96.38(0.02)
Carbohydrate (g)	190.69	116.63-264.74	0.25	0.05-0.44	360.18(98.68)	280.11(24.50)
Fiber (g)	11.02	4.73-17.30	0.32	0.15-0.49	33.88(8.14)	21.86(2.61)
Calcium (mg)	505.47	263.40-747.53	0.31	0.11-0.51	1094.48(319.90)	844.66(99.15)
Iron (mg)	9.09	5.26-12.92	0.30	0.10-0.50	17.67(5.66)	14.44(1.71)
Potassium (mg)	2130.28	1034.05-3226.51	0.28	0.04-0.51	4303.99(1100.15)	3314.98(302.82)
Zinc (mg)	13.13	1.51-24.76	0.25	0.17-0.33	41.46(13.55)	23.48(3.38)
Vitamin C (mg)	70.35	16.55-124.16	0.15	0.21-0.28	250.50(11.79)	107.48(1.75)
Vitamin A (IU)	5303.30	1899.67-8.706.94	0.17	-0.05-0.40	12330.39(2550.22)	7443.36(442.61)
Vitamin E (mg)	5.21	0.04-10.38	0.11	-0.42-0.64	8.72(1.85)	6.21(0.21)
Selenium (mg)	103.25	71.99-134.51	0.14	-0.05-0.33	143.71(47.19)	123.37(6.61)

• Note:
• ^*
  Calibration factor;
• ^**
  Analysis performed on log-transformed nutrients and adjusted for energy;
• ^***
  Gross value;
• ^****
  FFQ calibrated by linear regression , with 24hDR (energy and log-transformed nutrients, adjusted for energy and de-attenuated ) as the dependent variable and the FFQ as an independent variable; (E) Energy; (PTN) Protein; (LIP) Lipid; (CHO) Carbohydrate; (FI) Fiber; (CA) Calcium; (FE) Iron; (K) Potassium; (ZN) Zinc; (SE) Selenium. CI: Confidence Interval; FFQ2: Food Frequency Questionnaire 2; SD: Standard Deviation.

DISCUSSION

In the present study, reproducibility, validity and calibration factors were estimated to correct food intake data obtained through the FFQ adapted for application in populations of German descendants living in a city in southern Brazil.

Reproducibility checks whether results are similar on two or more different measurement occasions; it refers, therefore, to their answers agreement [¹³]. In terms of reproducibility, in this work, the ICC analysis of energy and nutrients pointed to a positive correlation between the results obtained at the two points in time of FFQ data collection, results that indicate an acceptable reproducibility of the instrument for most of the nutrients tested (except for the raw data for vitamin E and for the adjusted data for vitamin C). Sam et al. [¹⁴] and Molina et al. [⁸], after performing nutrient adjustments for energy, found that the correlation coefficients between the results of the two FFQ collections decreased. However, corroborating the results found in this study, Bonatto et al. [¹⁵] identified an increase in the ICCs of nutrients when corrected for energy.

Some authors mention that CCI values between 0.40 and 0.70 indicate acceptable reproducibility of the tested instrument [⁷,¹³]. Additionally, Beaton et al [¹³] warn that high reproducibility values can occur when the respondent repeats the same error. In order to mitigate this bias when investigating the reproducibility of the FFQ, the time interval between data collections should not be abusively short or long [⁷]. In the present study, it is believed that this aspect did not interfere with the results, since the time interval between FFQ applications was three months. However, changes in food intake itself may have occurred, especially in the nutrients that showed a poor correlation. In addition, age, socioeconomic status, ability to estimate food intake in a reliable form, and difficulty in understanding the complexity of the assessment instrument can also affect the reproducibility of the method [¹⁶].

In this study, to test the relative validity of the FFQ, different approaches were used. The application of the ICC indicated a weak correlation between FFQ2 and the mean of the 24hDRs, with an ICC below 0.40 for all nutrients tested before and after correction for energy. In another FFQ validation study, which adopted the 24hDR as a reference method (retrospective or prospective), it was shown, in general, superior correlations [¹⁷]. It was also observed that the correlation values for reproducibility identified here were higher than those for validity; these findings are similar to those of other works [⁸,¹⁵].

Assessing the acceptable degree of accuracy of an FFQ largely depends on the purpose of the questionnaire. For studies of association between dietary exposure and an outcome of interest, correlation coefficients lower than 0.3 or 0.4 seem inadequate to obtain associations [⁷]. On the other hand, Hinnig et al. [¹⁸] mention that high correlation coefficients may not indicate method validity, as the correlation may be due to similar errors presented by both methods. The ideal is that the reference method does not present errors correlated with the tested instrument [¹⁹]. However, the 24hDR (method adopted to test the validity in this work) depends, as well as the FFQ, on the respondents’ memory and their awareness of the food portion size [⁵]. Although the presence of correlated error is acknowledged, the 24hDR is often applied in validation studies of food surveys, since it is easy to apply, low cost and sensitive to culturally specific groups, such as the population targeted in this study [⁵,²⁰]. The food recording method consisting of food weighing has been considered the best method for FFQ validation studies; however, when the education level and the motivation level do not allow the use of the instrument, the 24hDR has been considered the “gold standard” method [¹⁹]. It is worth remembering that, while the FFQ produces measures of habitual intake, the application of a single 24hDR does not represent the individual’s usual intake [¹⁸]. This limitation is due to the high variability in the consumption of nutrients by the same person (intrapersonal) and between persons (interpersonal); aspects that may have influenced the validation results found here, since the 24hDR may not represent the individual’s usual consumption.

According to Fisberg et al. [¹⁶], the credibility of the standard method adopted depends on the number of people evaluated (interpersonal variability), the number of times the method is repeated (intrapersonal variability), the population participating in the study and the nutrients studied. In this study, the recommendation of the number of repetitions of the reference method (three times) was respected, as well as the size of the study population, although this size was very close to the recommended minimum, due to the low adherence of the first volunteers who participated in the population-based study (Study of Living Conditions and Health of the Population of Pomerode, SC) [⁷]. Aspect that amplifies the interpersonal variability, contributing to a lower correlation between the methods. However, with the number of data available for validation evaluation, it was estimated that it would be possible to identify mean differences of 10 points between the groups, with a standard deviation of 15 and 9 points, respectively, considering a power of 80% and a 95% confidence level. In addition, to alleviate the low adherence of volunteers to this survey, adjustments were made for intrapersonal variability (de-attenuation) in an attempt to bring the food intake of the reference method closer to the usual consumption. However, after nutrient adjustment, there was a reduction in the coefficients. It is possible that the nutrients variability is due to systematic errors of food intake under- or over-reporting, which are common to food intake assessment methods [²¹].

Another approach adopted to test the instrument’s validity was the energy and nutrient intake distribution tertiles. An adequate percentage of agreement was identified between the FFQ2 and 24hDR instruments and a low percentage of disagreement. The performance of the kappa-weighted statistical test showed results that approached 0.21 or 0.40, pointing to a weak to moderate agreement [²²,²³]. Better results were found in a study that evaluated the relative validity of an image-supported FFQ [²⁴].

It should be noted that in studies that seek to assess the diet-disease relationship, food intake is often categorized to estimate association measures. Thus, the ability of a questionnaire to correctly classify individuals according to their level of intake (low, medium or high) allows us to verify which individuals are at higher risk for a given disease, depending on the consumption observed. This is why the FFQs, although they have limitations, remain until today the most used method of food assessment to study eating patterns and population behaviors [⁵,²⁵].

Another methodology used to analyze the FFQ in this study was proposed by Bland & Altman [¹¹], which assesses agreement between instruments through consumption intervals. In this method, the scatter plot associates the difference between the measurements (Y axis) and their mean (X axis), if there is agreement between the methods and if the points all fall on the 0 value of the Y axis. In the case of differences, the points are randomly distributed around the 0 value of the Y axis [¹¹], a trend found in this study for most nutrients, in addition to the overestimation of energy and nutrient consumption indicated by FFQ2 in relation to the 24hDR, which indicates disagreement between the methods by this approach. Overestimation of nutrient intake using the FFQ in adults is a common finding in the literature, which may be related to inaccuracy in reporting the frequency of consumption or portion [⁸,²⁶]. The 24hDR values may appear underestimated when the parameter is not applied on a weekend day, since food intake on the weekend is usually significantly higher than on other days of the week [¹³,²⁶]. However, in the present study, Sunday was one of the days evaluated by the 24hDR.

Another approach taken in this study was the estimation of FFQ2 calibration factors and the correction of data by the reference method, mainly necessary for questionnaires that will be used in studies that assess the diet-disease relationship [²⁷]. The absence of bias in the questionnaire, represented by a calibration factor close to 1, indicates that the mean intake estimated through the questionnaire is similar to the mean estimated using the reference method [²⁸]. The highest calibration factors found in this study were 0.32 for fiber and 0.31 for calcium. According to Voci et al. [²⁸], low calibration factors may be related to random FFQ errors. In this study, the mean values and standard deviations of energy and nutrients in the FFQ after its calibration were smaller than the uncalibrated ones, which suggests an increase in the accuracy of the estimate, that is, the calibrated data have less variation than the original data. It is noteworthy that calibration factors located far from 1.0 may be associated with some factors, such as: time elapsed between data collection using the reference method (one month between each 24hDR), number of repetitions of the 24hDR, small group sample due to low adherence (minimum recommended in the literature), age of the participant, as we deal with a predominantly older population, and/or collection performed by different evaluators, which may result in disagreement in data collection, despite the evaluators having been trained.

CONCLUSION

The FFQ evaluated in this study showed good reproducibility. In the validity analysis, weighing the ICC values, the FFQ showed a weak correlation for most nutrients, except for crude zinc. Considering the degree of agreement by weighted kappa, the instrument indicated weak to moderate agreement for most nutrients and in the distribution in tertiles of energy and nutrient intake, an adequate degree of agreement was identified between the instruments. In this study, there was a reduction in the mean values of energy and nutrients in FFQ2, after data calibration, approaching the values obtained in the reference method. However, the calibration factors were low, probably due to the possible limitations of this study, such as: the small sample size obtained, which possibly limited the statistical power of the evaluated instrument, the different nature of the two food intake assessment instruments (FFQ and 24hDR) and the low adherence of the first volunteers in the population-based study that gave rise to this study.

Although the study results do not confirm reasonable validity values, the study offers data for future work, this being the first study to verify the reproducibility and validity of a quantitative FFQ developed specifically for German descendants residing in Brazil.

• Support: Programa Institucional de Bolsas de Iniciação Científica, Universidade Regional de Blumenau (PIBIC/FURB) (Ca-se No. 459/2014).
• Article derived from the dissertation by G CHIARELLI, entitled “Validação de um Questionário de Frequência Alimentar para a População de Pomerode, SC”, Universidade Regional de Blumenau, 2015.

How to cite this article

• Chiarelli G, Hölfelmann DA, Silveira JLGC, Alves MU, Azevedo LC. Validity and reproducibility of a Food Frequency Questionnaire for German descendants living in Brazil. Rev Nutr. 2021;34:e200048. https://doi.org/10.1590/1678-9865202134e200048

REFERENCES

• ¹
  Romani PF, Strey MN. Hábitos alimentares, saúde e cultura: o olhar de profissionais da nutrição em uma cidade de colonização italiana. RCO. 2019;1:66-86. https://doi.org/10.25112/rco.v1i0.1490
  » https://doi.org/10.25112/rco.v1i0.1490
• ²
  Relly E. Imigração alemã ao Brasil (século XIX) e Prússia: fronteiras permeáveis e diálogos entre história global e micro-história. Hist Unisinos. 2016;20:273-86. https://doi.org/10.4013/htu.2016.203.04
  » https://doi.org/10.4013/htu.2016.203.04
• ³
  Monasterio L. Surnames and ancestry in Brazil. Plos One. 2017;12:e0176890. https://doi.org/10.1371/journal.pone.0176890
  » https://doi.org/10.1371/journal.pone.0176890
• ⁴
  Franzen DO, Badalotti C, Chaves GC. Dimensões da cultura germânica em Itapiranga (SC): o Patrimônio imaterial e sua relação com a identidade, a Memória e a tradição. Rev Hist Soc Ment. 2019;23:171-88. https://doi.org/10.35588/rhsm.v23i1.3439
  » https://doi.org/10.35588/rhsm.v23i1.3439
• ⁵
  Naska A, Lagiou A, Lagiou P. Dietary assessment methods in epidemiological research: current state of the art and future prospects. F1000 Res. 2017;6:926. https://doi.org/10.12688/f1000research.10703.1
  » https://doi.org/10.12688/f1000research.10703.1
• ⁶
  Martín-Calvo N, Martínez-González MA. Controversy and debate: memory-based dietary assessment methods paper 2. J Clin Epidemiol. 2018;104:125-9. https://doi.org/10.1016/j.jclinepi.2018.08.005
  » https://doi.org/10.1016/j.jclinepi.2018.08.005
• ⁷
  Cade J, Thompson R, Burley V, Daniel Q. Development, validation and utilization of food-frequency questionnaires-a review. Public Health Nutr. 2002;5:567-87. https://doi.org/10.1079/PHN2001318
  » https://doi.org/10.1079/PHN2001318
• ⁸
  Molina MCB, Benseñor IM, Cardoso LO, Velasquez-Melendez G, Drehmer M, Pereira TSS, et al Reprodutibilidade e validade relativa do Questionário de Frequência Alimentar do ELSA-Brasil. Cad Saúde Pública. 2013;29:379-89. https://doi.org/10.1590/S0102-311X2013000200024
  » https://doi.org/10.1590/S0102-311X2013000200024
• ⁹
  Da Rosa AC, Schmitz B, Chiarelli G, Silveira JLGC, Alves MU, Azevedo Campanella LC. Adaptação de um inquérito alimentar a ser utilizado em um estudo longitudinal a partir do contexto sociocultural de uma população de colonização alemã. Rev Atenção Saúde. 2016;14:34-41. https://doi.org/10.13037/ras.vol14n50.3828
  » https://doi.org/10.13037/ras.vol14n50.3828
• ¹⁰
  Willett W, Stampfer MJ. Total energy intake: implications for epidemiological analyses. Am J Epidemiol. 1986;124:17-27. https://doi.org/10.1093/oxfordjournals.aje.a114366
  » https://doi.org/10.1093/oxfordjournals.aje.a114366
• ¹¹
  Bland JM, Altman DG. Measuring agreement in method comparison studies. Stat Methods Med Res. 1999;8:135-60. https://doi.org/10.1177/096228029900800204
  » https://doi.org/10.1177/096228029900800204
• ¹²
  Kaaks R, Riboli E, Van SW. Calibration of dietary intake measurements in prospective cohort studies. Am J Epidemiol. 1995;142:548-56. https://doi.org/10.1093/oxfordjournals.aje.a117673
  » https://doi.org/10.1093/oxfordjournals.aje.a117673
• ¹³
  Beaton GH, Burema J, Ritenbaugh C. Errors in the interpretation of dietary assessments. Am J Clin Nutr. 1997;65:1100-007. https://doi.org/10.1093/ajcn/65.4.1100S
  » https://doi.org/10.1093/ajcn/65.4.1100S
• ¹⁴
  Sam CHY, Skidmore P, Skeaff S, Parackal S, Wall C, Bradbury KE. Relative validity and reproducibility of a short food frequency questionnaire to assess nutrient intakes of new Zealand adults. Nutrients. 2020;12:619. https://doi.org/10.3390/nu12030619
  » https://doi.org/10.3390/nu12030619
• ¹⁵
  Bonatto S, Henn RL, Olinto MTA, Anjos LA, Wahrlich V, Waissmann W. Reprodutibilidade, validade relativa e calibração de um questionário de frequência alimentar para adultos da Região Metropolitana de Porto Alegre, Rio Grande do Sul, Brasil. Cad Saúde Pública. 2014;30:1837-48. https://doi.org/10.1590/0102-311X00151313
  » https://doi.org/10.1590/0102-311X00151313
• ¹⁶
  Fisberg RM, Slater B, Marchioni DML, Araújo L. Martini Inquéritos alimentares: métodos e bases científicas. São Paulo: Manole; 2005.
• ¹⁷
  Rodríguez MG, Saldaña MR, Leyva JMA, Rojas RM, Recio GM. Design and validation of a food frequency questionnaire (FFQ) for the nutritional evaluation of food intake in the Peruvian Amazon. J Health Popul Nutr. 2019;38:47. https://doi.org/10.1186/s41043-019-0199-8
  » https://doi.org/10.1186/s41043-019-0199-8
• ¹⁸
  Hinnig PF, Prado BG, Latorre MRDO. Validity and reproducibility of a food frequency questionnaire for children. J Hum Growth Dev. 2018;28:120-8. https://doi.org/10.7322/jhgd.147217
  » https://doi.org/10.7322/jhgd.147217
• ¹⁹
  Biró G, Hulshof KF, Ovesen L, Amorim CJA. Selection of methodology to assess food intake. Eur J Clin Nutr. 2002;56:25-32. https://doi.org/10.1038/sj.ejcn.1601426
  » https://doi.org/10.1038/sj.ejcn.1601426
• ²⁰
  El Kinany K, Garcia-Larsen V, Khalis M, Deoula MMS, Benslimane A. Adaptation and validation of a food frequency questionnaire (FFQ) to assess dietary intake in Moroccan adults. Nutr J. 2018;17:61. https://doi.org/10.1186/s12937-018-0368-4
  » https://doi.org/10.1186/s12937-018-0368-4
• ²¹
  Machado CH, Lopes ACS, Santos LCD. Notificação imprecisa da ingestão energética entre usuários de Serviços de Promoção à Saúde. Ciênc Saúde Coletiva. 2017;22:417-26. https://doi.org/10.1590/1413-81232017222.21492015
  » https://doi.org/10.1590/1413-81232017222.21492015
• ²²
  Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159-74. https://doi.org/10.2307/2529310
  » https://doi.org/10.2307/2529310
• ²³
  Miot HA. Análise de concordância em estudos clínicos e experimentais. J Vasc Bras. 2016;15:89-92. https://doi.org/10.1590/1677-5449.004216
  » https://doi.org/10.1590/1677-5449.004216
• ²⁴
  Brito AP, Araujo MC, Guimarães CP, Pereira RA. Validade relativa de questionário de frequência alimentar com suporte de imagens. Ciênc Saúde Coletiva. 2017;22:457-68. https://doi.org/10.1590/1413-81232017222.26392015
  » https://doi.org/10.1590/1413-81232017222.26392015
• ²⁵
  Aoun C, Daher RB, El Osta N, Papazian T, Khabbaz LR. Reproducibility and relative validity of a food frequency questionnaire to assess dietary intake of adults living in a Mediterranean country. Plos One. 2019;14:e0218541. https://doi.org/10.1371/journal.pone.0218541
  » https://doi.org/10.1371/journal.pone.0218541
• ²⁶
  Zack RM, Irema K, Kazonda P, Leyna GH, Liu E, Gilbert S, et al Validity of a Food Frequency Questionnaire to Measure Nutrient and Food Intake in Tanzania: Barriers and Opportunities for, 40. Public Health Nutr. 2018;21:2211-20. https://doi.org/10.1017/S1368980018000848
  » https://doi.org/10.1017/S1368980018000848
• ²⁷
  Lopes RDVC, Teixeira JA, Marchioni DM, Villa LL, Giuliano AR, Fisberg RM. Improvement in dietary intake estimates through the combined use of different approaches. Rev Nutr. 2019;32:e180137. https://doi.org/10.1590/1678-9865201932e180137
  » https://doi.org/10.1590/1678-9865201932e180137
• ²⁸
  Voci SM, Slater B, Silva MV, Marchioni DML, Latorre MRDO. Estudo de calibração do Questionário de Frequência Alimentar para Adolescentes (QFAA). Ciênc Saúde Coletiva. 2011;16:2335-43. https://doi.org/10.1590/S1413-81232011000400033
  » https://doi.org/10.1590/S1413-81232011000400033

Publication Dates

• Publication in this collection
  17 Dec 2021
• Date of issue
  2021

History

• Received
  11 May 2020
• Reviewed
  28 May 2021
• Accepted
  27 July 2021