Decision flowchart for food classification by the extension and purpose of industrial processing: update and practical application

Fluxograma de decisão para classificação de alimentos conforme a extensão e propósito do processamento industrial: atualização e aplicação prática

ABSTRACT

Objective

To describe the development and update of an instrument for food categorisation according to the extension and purpose of industrial processing, and to test its practical application.

Methods

After updating the instrument based on a recent publication on the NOVA classification, it was applied by five researchers to a database of 108 food items. These items are part of a database of foods announced in the health-related sections of supermarket promotional circulars. The Cohen’s weighted kappa coefficient was calculated to determine intra-rater agreement; Fleiss’ kappa and Kendall’s coefficient were applied to determine inter-rater agreement.

Results

In the updated version, two classes of additives and eight substances considered by the most recent publication as specific to ultra-processed foods were added. The intra-rater agreement was 100% (p<0.001), indicating an “almost perfect” agreement; Fleiss’ agreement among all raters ranged from 74% to 97% (p<0.001), which represents an agreement that ranged from “strong” to “almost perfect”; Kendall’s W was higher than 0.93 (p<0.001) among all raters.

Conclusion

The updated instrument showed high agreement and proved to be a methodologically sound and applicable tool for the purpose of classifying foods by the extension and purpose of industrial processing.

Keywords
Classification; Flowchart; Food processing; Healthy eating

RESUMO

Objetivo

Descrever o desenvolvimento e atualização de um instrumento para categorização dos alimentos de acordo com a extensão e propósito de seu processamento industrial e avaliar sua aplicação prática.

Métodos

Após a atualização do instrumento com base em publicação recente sobre o tema, ele foi aplicado por cinco pesquisadores em um banco de dados de 108 alimentos. Esses alimentos são parte de um banco de dados de alimentos anunciados em seções relacionadas à saúde de panfletos promocionais de supermercado. Para avaliar a concordância intra-avaliador foi calculado o kappa ponderado de Cohen e para a concordância interavaliadores foram calculados o kappa de Fleiss e o coeficiente de concordância de Kendall.

Resultados

Na versão atualizada foram adicionadas duas classes de aditivos e oito substâncias, consideradas pela publicação mais recente como específicas de alimentos ultraprocessados. A concordância intra-avaliador foi de 100% (p<0,001), o que indica uma concordância “quase perfeita”; a concordância de Fleiss entre todos os avaliadores variou de 74% a 97% (p<0,001), o que representa uma concordância que variou de “forte” a “quase perfeita”; a concordância de Kendall foi W >0,93 (p<0,001) entre todos os avaliadores.

Conclusão

O instrumento atualizado apresentou uma elevada concordância e mostrou-se uma ferramenta metodologicamente útil e aplicável quando se tem por objetivo classificar alimentos pela extensão e propósito do processamento industrial.

Palavras-chave
Classificação; Fluxograma; Processamento de alimentos; Alimentação saudável

INTRODUCTION

The industrial processing of food and its impact on individual health is a relevant topic of debate in both scientific and lay communities, and a concern about the replacement of most freshly prepared meals with ultra-processed foods is usually part of it [¹-⁴]. Therefore, when compared to previous years, food processing grew as a relevant determinant of food quality and diseases, being applied to studies of food availability, diet quality, and health outcomes [³,⁵].

In Brazil, the last decades witnessed political, social, economic, and cultural transformations that impacted the population’s health and standards of food consumption. Consequently, in 2011, the Ministério da Saúde (MS, Ministry of Health) began the elaboration of a new edition of the Guia Alimentar para a População Brasileira (GAPB, Dietary Guidelines for the Brazilian Population), following the World Health Organization’s (WHO) recommendation that guidelines for adequate and healthy eating be updated periodically [⁶]. In its most recent edition, GAPB’s guidelines are based on food classification according to the extension and purpose of its industrial processing before acquisition [⁴]. Such classification was later updated and named NOVA [⁷].

As the extension and purpose of food’s industrial processing grew as a relevant marker of the population’s diet quality and health, national and international nutritional researchers began applying the NOVA method to assess the nutritional quality of food [⁸-¹⁴]. According to the guidelines of the United Nations’ Food and Agriculture Organisation (FAO) for the collection of information on food processing, the list of ingredients must be used whenever possible for a clearer and more reliable classification [¹]. The Brazilian legislation makes it mandatory that the list of ingredients is found on the label of foods that are packaged away from the consumers, except for single-ingredient foods [¹⁵]. Following FAO’s recommendation is thus made easier when the method of data collection allows access to the label.

However, such classification still faces practical obstacles, such as those described as follows: which additive(s) make a food item considered ultra-processed? Do ingredients absent from traditional preparations, such as lactose, gluten, protein isolates, and sweeteners, cause an item to move into the ultra-processed group of food? May replacing one ingredient, like vegetable fat with butter, change an item’s classification from processed to ultra-processed food? A lot of helpful information to solve these and other doubts is found in earlier publications, but not in a systematised fashion [⁴,⁷,¹⁶,¹⁷]. Thus, each question imposes revising different documents.

To systematise and standardise the process of classification by the extension and purpose of industrial processing, a decision flowchart was developed based on documents about the NOVA classification that were published until 2016 [⁴,⁷,¹⁶,¹⁸]. The flowchart gathers the information contained in these publications and has been used in previous studies on food choices in supermarkets and on the items advertised in the health-related sections of supermarket circulars [¹⁸,¹⁹]. Additional criteria to classify food according to NOVA are presented in a 2019 publication [¹⁷]. Therefore, it is necessary to update the flowchart [¹⁸]. Thus, the present study aims to describe the development and update of a flowchart for classifying food according to the extension and purpose of industrial processing, as well as to evaluate the flowchart’s practical application.

METHODS

Initially, all the publications produced by the NOVA development team on the extension and purpose of food’s industrial processing until 2016 were identified and read. Three papers contained all the necessary and indispensable information and thus were used for the elaboration of the decision flowchart [⁴,⁷,¹⁶].

Questions and answers were suggested to build the flowchart’s pathway leading the food item to its classification. The initial questions were tested in different orders, and the need for including a question or inverting the order between questions was verified until the final model.

The final model was submitted to quality control by Cohen’s kappa coefficient calculation, resulting in values from 0.94 to 1.00 between food categories and 0.99 as general punctuation, indicating an “almost perfect” agreement among the researchers [¹⁸,²⁰].

The flowchart’s update was based on the 2019 publication, and additional information was identified independently by two rresearchers [¹⁷]. As no divergence was found, all the information identified was included.

The NOVA classification categorises food into four groups: 1) Unprocessed or minimally processed foods; 2) Processed culinary ingredients; 3) Processed foods; 4) Ultra-processed foods [⁷].

Unprocessed foods are the edible parts of plants or animals, algae, mushrooms, and water. When these items have been pasteurised, fractioned, dried, ground, vacuum-packed, or subjected to non-alcoholic fermentation, without the addition of processed culinary ingredients (salt, sugar, oils, among others) or others of exclusive industrial use, they are classified as minimally processed foods. Unprocessed and minimally processed foods form Group 1 in the classification [⁴,⁷]. Also included in this group are food items with similar characteristics but with additives not defined by NOVA as exclusive to ultra-processed foods [⁷,¹⁷].

Processed culinary ingredients are obtained by processes of extraction from unprocessed foods (e.g., sugar, butter) or nature (e.g., salt, honey) [⁴,⁷]. Food items composed of more than one ingredient in this group remain in the same group (e.g., salted butter). Those with additives that are not exclusive to ultra-processed foods according to the NOVA classification are also included [⁷,¹⁷]. An exception is made for anti-humectants, which are allowed in salt without altering its classification [⁷].

Processed foods include industrially processed items and items composed of foods from the first group and processed culinary ingredients [⁴,⁷]. Food items that contain additives or other ingredients which nonetheless have not been defined by NOVA [⁷,¹⁷] as exclusive to ultra-processed foods are included in this group. Alcoholic beverages obtained from the alcoholic fermentation of first-group foods, such as wine, beer, and cider, and without further distillation processes, are also in Group 3 [⁷].

Ultra-processed foods undergo several processing stages and techniques that contain exclusively industrial ingredients and substances [⁴,⁷]. These include foods with the following additives: carbonation agents; bulking agents; firming agents; anti-bulking, anti-caking and anti-humectants agents; de-foaming agents; flavours, flavour enhancers; dyes and other colours; non-sugar sweeteners; emulsifiers, emulsifying salts; thickeners; foaming agents; colour stabilisers; gelling agents; glazing agents; nitrate or nitrite; sequestrants; humectants. There is no specification related to the use of natural dyes and flavours; thus, in the flowchart, those are considered as of specific use to ultra-processed foods [⁷,¹⁷]. Sausages and similar processed meats, which are characterised by the addition of nitrates and nitrites, are also in this group [⁴,⁷].

Group 4 also includes food items like vegetable fat, hydrogenated or interesterified oils, and others that may contain trans fats, invert sugar, sweeteners, dextrose, fructose, glucose, lactose, maltodextrin, concentrated fruit juice, syrup, modified starch, casein, protein isolates, whey protein, whey, soluble or insoluble fibre, or gluten [⁴,⁷,¹⁷]. Other foods classified in the ultra-processed food group have undergone or had one of its ingredients undergoing exclusively industrial processing techniques, such as extrusion, moulding, pre-frying, hydrogenation, or hydrolysis [⁴,⁷,¹⁶]. The processing technique is not necessarily informed on the food labels, making it essential to research the item’s production in case of uncertainty. Alcoholic beverages processed by further distillation process (whisky, cachaça, vodka) are also included in this group [⁷].

One of the questions in the decision flowchart is, “does the food item contain more than one ingredient?”. The correct application of the flowchart implies disregarding water and additives as ingredients. Thus, if a food item is composed of vinegar, water, and antioxidant agents, it is understood as a one-ingredient food item [¹⁸].

When additives’ functions are not described in the ingredient list, as mandated by the Brazilian legislation, one must look for it in the legislation or equivalent products. Whenever the function of an additive in an item cannot be clearly defined, the conservative criterion is suggested, keeping the item in the group of lower extension and purpose of industrial processing [¹¹,¹⁵,²¹].

To verify the practical application of the decision flowchart, 25% (n=108) of the products of a databank from a previous study were randomly selected with the programme Research Randomizer^® [¹⁹]. Those were mostly cookies, fish, vegetable milk, and dairy beverages. To assess intra-rater agreement, a researcher applied the updated flowchart to these products with a system of numeric coding from 1 to 4, which corresponded to the four categories of NOVA classification. After 45 days, the procedure was repeated by the same researcher and the two classifications were compared. To evaluate inter-rater agreement, five researchers (two of whom were not involved in the elaboration and update of the flowchart) applied the tool independently following the same procedure, and their classifications were compared [²²,²³]. All the researchers were nutritionists with different levels of familiarity of the topic – a Ph.D. holder and two Ph.D. candidates in Nutrition involved in elaborating the flowchart and with experience in using the NOVA classification in many studies, and two master’s degree candidates in Nutrition, one of whom did not have previous knowledge of the method and the other who applied the decision flowchart in her master’s thesis.

Cohen’s Weighted Kappa coefficient (k_w) was calculated to classify intra-rater agreement [²⁴,²⁵]. For inter-rater agreement (among all the raters; between the raters who worked on developing the flowchart; and between a developer of the flowchart and two researchers who were not involved), Fleiss’s Kappa (k) and Kendall’s (W) coefficient were calculated. Kendall’s W coefficient considers that wrongly placing a Group 4 food in Group 1 is graver than classifying a Group 4 food into Group 3 [²⁶]. Intra- and inter-rater agreement was classified according to Landis and Koch [²⁰]. The programme Stata version 13.0 (StataCorp, College Station, Texas, USA) was used for the data analysis. A p<0.05 indicates that agreement is not due to chance.

RESULTS

For the update of the “Decision flowchart for classifying food items according to the extension and purpose of industrial processing” (Figure 1 – Portuguese version; Figure 2 – English version), new information was included in the third and fourth questions (“Does the food contain additives that simulate sensorial properties, modifying the final product’s colour, smell, taste, or texture?” and “Does it contain added substances such as...?”, respectively).

Figure 1
Decision flowchart for food classification by the extension and purpose of industrial processing.

Figure 2
Decision flowchart for food classification by the extension and purpose of industrial processing.

Two functions of additives were included in the third question, which are described as “thickener” (number 10) and “gelling agents” (number 13). “Emulsifying salts” and “emulsifier” were also included (number 9).

In the fourth question, the following substances were included, as cited: hydrogenated oils; sweeteners; concentrated fruit juice; fructose; dextrose; soluble and insoluble fibres, and whey protein. The terms “hydrolysed protein” and “soy protein isolate”, present in the previous version, were replaced by “protein isolates” [¹⁸].

Question number 7 of the previous version, “Does it contain salt, sugar, vinegar, oil, or other substances?” [¹⁸], was also updated to make it clearer what “substances” referred to. In the current version, it is presented as: “Does it contain salt, sugar, vinegar, oil, or other culinary processed ingredients?”.

Intra-rater agreement regarding the flowchart’s categories was 100% (k_w = 1.00, p<0.001, CI (95%) 0.83-1.17), indicating an “almost perfect” agreement [²⁰].

Fleiss’s agreement among all raters (Table 1) was above 90% for categories 1, 2, and 4 of extension and purpose of industrial processing (k=0.91-0.97; p<0.001) and 74% for Group 3 (k=0.74; p<0.001), representing an “almost perfect” agreement for categories 1, 2, and 4, and a “strong” agreement for category 3 [²⁰]. Among the raters involved in the development of the flowchart, agreement was equal to or above 90% for all categories (k=0.90-0.99; p<0.001), an “almost perfect” agreement [²⁰].

Table 1
Results of inter-rater agreement with Fleiss’s kappa. Florianópolis (SC), Brazil, 2021.

Category	A			B			C
	k	p		k	p		k	p
Unprocessed or minimally processed	0.93	<0.001		0.97	<0.001		0.90	<0.001
Processed culinary ingredient	0.91			0.90			0.90
Processed	0.74			0.93			0.65
Ultra-processed	0.97			0.99			0.97

• Note: p<0.05: Statistical significance. A: All five raters; B: Three raters who developed the flowchart; C: One rater who developed the flowchart and two rater who did not do so. k: Fleiss’s kappa value.

When the agreement between a rater involved in developing the flowchart and other raters not involved in it were compared, an agreement equal to or above 90% was found for categories 1, 2, and 4 of extension and purpose of industrial processing (k=0.90-0.97; p<0.001) and 65% for Group 3 (k=0.65; p<0.001), representing “almost perfect” agreement for categories 1, 2, and 4, and “strong” agreement for category 3 [²⁰].

Kendall’s W was higher than 0.93 (p<0.001) among all evaluators (Table 2).

Table 2
Results of inter-rater agreement with Kendall’s W coefficient. Florianópolis (SC), Brazil, 2021.

Rater	1			2			3			4
	W	p		W	p		W	p		W	p
2	0.98	<0.001
3	0.99	<0.001		0.98	<0.001
4	0.95	<0.001		0.94	<0.001		0.96	<0.001
5	0.95	<0.001		0.94	<0.001		0.95	<0.001		0.94	<0.001

• Note: p<0.05: Statistical significance. Raters 1, 2, and 3: Developers of the flowchart; Raters 4 and 5: Not developers of the flowchart. W: Kendall’s coefficient.

DISCUSSION

The study aimed to describe the development, to update, and to assess the practical application of the decision flowchart for classifying food items according to the extension and purpose of industrial processing.

The flowchart was applied independently and in a satisfactory fashion by five researchers, involved or not with the development of the instrument. It obtained a level of agreement kappa which varied from “strong” to “almost perfect”; a Kendall’s W above 0.90 among the researchers; and a 100% kappa intra-rater agreement [²⁰,²⁶]. Thus, one may conclude that the flowchart is an applicable and objective method for categorising foods by the extension and purpose of industrial processing, excluding the need for deep previous knowledge on the NOVA classification since the researcher’s interpretation is didactically directed by the questions presented in the instrument.

However, the adequate use of the flowchart requires access to the list of ingredients of the food items that will be classified, thus allowing a clearer and more trustworthy classification [¹]. We did not identify other specific methods for applying the NOVA classification [⁷] based on the ingredient list. Using the list of ingredients as suggested by FAO [¹] is a strong point of the flowchart, as a systematically organised methodology may reduce disagreements in the classification and consequently reduce the underestimation of ultra-processed items. Such underestimation may occur when the list is not available for access and when methodological strategies based on previous knowledge from researchers and that consider a standard formula for an item are used [⁸-¹¹,¹⁴,²⁷]. We highlight the importance of classifying food in a more trustworthy way, especially ultra-processed ones, as their increased consumption has been associated to several diseases like obesity, cardiovascular disease, diabetes mellitus, and metabolic syndrome [²⁸,²⁹].

The present study’s decision flowchart was developed to be used by nutrition professionals, as well as in scientific environments. Thus, studies and public campaigns to make the identification of ultra-processed foods by the general population easier are needed, given that the literature has already established how hard it is for individuals to identify such products [³⁰]. Indirectly, however, the flowchart’s use by nutritionists and researchers may help raise awareness in the population.

Among the flowchart’s limitations is the need to access the list of ingredients. Thus, it is not recommended for studies that assess consumption and classification of culinary preparations. For those, we suggest using the decision tree developed by Louzada and collaborators [³¹]. Also, it may seem difficult to answer question number 5 of the flowchart, related to the use of industrial techniques, as this information is not present on the labels. However, in our practical experience, doubts on the use of industrial processing techniques in food items are easily solved with quick online searches. Moreover, the deep knowledge that three authors had on NOVA classification may have been a bias for the flowchart’s practical application. To minimise such bias, two researchers not involved with the flowchart’s elaboration also participated in the practical application. Thus, the instrument may be used adequately by researchers and nutritionists less familiar with the NOVA classification. However, to extrapolate the instrument’s level of agreement by nutritionists with clinical experience, studies with more robust samples are needed.

CONCLUSION

The updated flowchart for classifying food items presented elevated agreement among the researchers who applied it, constituting a methodologically sound and applicable tool for research aiming to classify food by the extension and purpose of industrial processing. Even after the inclusion of nutritional researchers with less experience in the NOVA classification, agreement was still high, indicating it may also be employed to subsidise nutritionists’ clinical practise and direct nutritional counselling to meet the recommendations of the Dietary Guidelines for the Brazilian Population.

ACKNOWLEDGMENTS

We thank the master’s degree candidates Lorena Cristina Guerreiro Gatinho (PPGN-UFSC) e Marina Padovan Luiz (PPG-CNEM-UNICAMP) for their assistance in classifying the food items with the flowchart.

• Support: Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (Capes) (Financial Code 001).

How to cite this article

• Botelho, AM, Camargo, AM, Mazzonetto, AC, Fiates, GMR. Decision flowchart for food classification by the extension and purpose of industrial processing: update and practical applicationTítulo. Rev Nutr. 2022;35:e210184. https://doi.org/10.1590/1678-9865202235e210184

REFERENCES

• ¹
  Food and Agriculture Organization of the United Nations. Guidelines on the collection of information on food processing through food consumption surveys. Rome: The Organization; 2015 [cited 2021 May 5]. Available from: http://www.fao.org/3/i4690e/i4690e.pdf
  » http://www.fao.org/3/i4690e/i4690e.pdf
• ²
  Pan American Health Organization of the World Health Organization. Ultra- processed food and drink products in Latin America: trends, impact on obesity, policy implications. Washington: The Organization; 2015 [cited 2021 Apr 26]. Available from: https://iris.paho.org/bitstream/handle/10665.2/7699/9789275118641_eng.pdf
  » https://iris.paho.org/bitstream/handle/10665.2/7699/9789275118641_eng.pdf
• ³
  Moubarac J-C, Parra DC, Cannon G, Monteiro CA. Food Classification Systems Based on Food Processing: significance and implications for policies and actions: a systematic literature review and assessment. Curr Obes Rep. 2014;3(2):256-272, https://doi.org/10.1007/s13679-014-0092-0
  » https://doi.org/10.1007/s13679-014-0092-0
• ⁴
  Ministério da Saúde (Brasil). Dietary Guidelines for the Brazilian Population. Brasília: Ministério; 2015. [cited 2020 Dec 2]. Available from: http://bvsms.saude.gov.br/bvs/publicacoes/dietary_guidelines_brazilian_population.pdf
  » http://bvsms.saude.gov.br/bvs/publicacoes/dietary_guidelines_brazilian_population.pdf
• ⁵
  Monteiro CA, Cannon G, Moubarac J-C, Levy RB, Louzada MLC, Jaime PC. The UN Decade of Nutrition, the NOVA food classification and the trouble with ultra-processing. Public Health Nutr. 2018;21(1):5-17, https://doi.org/10.1017/s1368980017000234
  » https://doi.org/10.1017/s1368980017000234
• ⁶
  World Health Organization. Global Strategy on Diet Physical Activity and Health. Geneva: Organization; 2004 [cited 2021 Mar 15]. Available from: https://www.who.int/dietphysicalactivity/strategy/eb11344/strategy_english_web.pdf
  » https://www.who.int/dietphysicalactivity/strategy/eb11344/strategy_english_web.pdf
• ⁷
  Monteiro CA, Cannon G, Levy R, Moubarac J-C, Jaime P, Martins AP, et al NOVA: the star shines bright. World Nutr. 2016;7(1-3):28-38.
• ⁸
  Martins APB, Levy RB, Claro RM, Moubarac JC, Monteiro CA, Martins APB, et al Participação crescente de produtos ultraprocessados na dieta brasileira (1987-2009). Rev Saude Publica. 2013;47(4):656-665, https://doi.org/10.1590/S0034-8910.2013047004968
  » https://doi.org/10.1590/S0034-8910.2013047004968
• ⁹
  Canella DS, Levy RB, Martins APB, Claro RM, Moubarac J-C, Baraldi LG, et al Ultra-processed food products and obesity in brazilian households (2008-2009). Plos One. 2014;9(3):e92752, https://doi.org/10.1371/journal.pone.0092752
  » https://doi.org/10.1371/journal.pone.0092752
• ¹⁰
  Louzada MLC, Martins APB, Canella DS, Baraldi LG, Levy RB, Claro RM, et al Ultra-processed foods and the nutritional dietary profile in Brazil. Rev Saude Publica. 2015;49(38):1-17. https://doi.org/10.1590/S0034-8910.2015049006132
  » https://doi.org/10.1590/S0034-8910.2015049006132
• ¹¹
  Steele EM, Baraldi LG, Louzada MLC, Moubarac J-C, Mozaffarian D, Monteiro CA. Ultra-processed foods and added sugars in the US diet: evidence from a nationally representative cross-sectional study. BMJ Open. 2016;6(3):e009892, http://dx.doi.org/10.1136/bmjopen-2015-009892
  » https://doi.org/10.1136/bmjopen-2015-009892
• ¹²
  Cediel G, Reyes M, Louzada MLC, Steele EM, Monteiro CA, Corvalán C, et al Ultra-processed foods and added sugars in the Chilean diet (2010). Public Health Nutr. 2018;21(1):125-133, https://doi.org/10.1017/s1368980017001161
  » https://doi.org/10.1017/s1368980017001161
• ¹³
  Marrón-Ponce JA, Sánchez-Pimienta TG, Louzada MLC, Batis C. Energy contribution of NOVA food groups and sociodemographic determinants of ultra-processed food consumption in the Mexican population. Public Health Nutr. 2018;21(1):87-93, https://doi.org/10.1017/s1368980017002129
  » https://doi.org/10.1017/s1368980017002129
• ¹⁴
  Rauber F, Steele EM, Louzada MLC, Millett C, Monteiro CA, Levy RB. Ultra-processed food consumption and indicators of obesity in the United Kingdom population (2008-2016). Plos One. 2020;15(5): e0232676. https://doi.org/10.1371/journal.pone.0232676
  » https://doi.org/10.1371/journal.pone.0232676
• ¹⁵
  Ministério da Saúde (Brasil). Resolução RDC no 259, de 20 de setembro de 2002. Aprova o regulamento técnico sobre rotulagem de alimentos embalados. Brasília: Diário Oficial da União; 2002.
• ¹⁶
  Monteiro CA, Levy RB, Claro RM, Castro IRR, Cannon G. A new classification of foods based on the extent and purpose of their processing. Cad Saude Publica. 2010;26:2039-2049. https://doi.org/10.1590/S0102-311X2010001100005
  » https://doi.org/10.1590/S0102-311X2010001100005
• ¹⁷
  Monteiro CA, Cannon G, Levy RB, Moubarac J-C, Louzada ML, Rauber F, et al Ultra-processed foods: what they are and how to identify them. Public Health Nutr. 2019;22(5):936-941. https://doi.org/10.1017/s1368980018003762
  » https://doi.org/10.1017/s1368980018003762
• ¹⁸
  Botelho AM, Camargo AM, Dean M, Fiates GMR. Effect of a health reminder on consumers’ selection of ultra-processed foods in a supermarket. Food Qual Prefer. 2019;71:431-437. https://doi.org/10.1016/j.foodqual.2018.08.017
  » https://doi.org/10.1016/j.foodqual.2018.08.017
• ¹⁹
  Botelho AM, Camargo AM, Medeiros KJ, Irmão GB, Dean M, Fiates GMR. Supermarket circulars promoting the sales of “healthy” foods: analysis based on degree of processing. Nutrients. 2020;12(9):e2877, https://doi.org/10.3390/nu12092877
  » https://doi.org/10.3390/nu12092877
• ²⁰
  Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159-174.
• ²¹
  Ministério da Saúde (Brasil). Resolução RDC no 45, de 03 de novembro de 2010. Dispõe sobre aditivos alimentares autorizados para uso segundo as Boas Práticas de Fabricação (BPF). Brasília: Diário Oficial da União; 2010.
• ²²
  Berchtold A. Test–retest: Agreement or reliability? Methodol Innov. 2016;9:e205979911667287. https://doi.org/10.1177%2F2059799116672875
  » https://doi.org/10.1177%2F2059799116672875
• ²³
  Kottner J, Audigé L, Brorson S, Donner A, Gajewski BJ, Hróbjartsson A, et al Guidelines for Reporting Reliability and Agreement Studies (GRRAS) were proposed. J Clin Epidemiol. 2011;64(1):96-106. https://doi.org/10.1016/j.jclinepi.2010.03.002
  » https://doi.org/10.1016/j.jclinepi.2010.03.002
• ²⁴
  Gisev N, Bell JS, Chen TF. Interrater agreement and interrater reliability: key concepts, approaches, and applications. Res Soc Adm Pharm. 2013;9(3):330-338. https://doi.org/10.1016/j.sapharm.2012.04.004
  » https://doi.org/10.1016/j.sapharm.2012.04.004
• ²⁵
  Mandrekar JN. Measures of Interrater Agreement. J Thorac Oncol. 2011;6(1):6-7. https://doi.org/10.1097/JTO.0b013e318200f983
  » https://doi.org/10.1097/JTO.0b013e318200f983
• ²⁶
  Legendre P. Species associations: the Kendall coefficient of concordance revisited. J Agric Biol Environ Stat. 2005;10(2):226-245. https://doi.org/10.1198/108571105X46642
  » https://doi.org/10.1198/108571105X46642
• ²⁷
  Rauber F, Louzada ML, Steele EM, Millett C, Monteiro CA, et al Ultra-Processed food consumption and chronic non-communicable diseases-related dietary nutrient profile in the UK (2008–2014). Nutrients. 2018;10(5):e587. https://doi.org/10.3390/nu10050587
  » https://doi.org/10.3390/nu10050587
• ²⁸
  Elizabeth L, Machado P, Zinöcker M, Baker P, Lawrence M. Ultra-Processed foods and health outcomes: a narrative review. Nutrients. 2020;12(7):1955. https://dx.doi.org/10.3390%2Fnu12071955
  » https://doi.org/10.3390%2Fnu12071955
• ²⁹
  Pagliai G, Dinu M, Madarena MP, Bonaccio M, Iacoviello L, Sofi F. Consumption of ultra-processed foods and health status: a systematic review and meta-analysis. Br J Nutr. 2021;125(3):308-318. https://doi.org/10.1017/s0007114520002688
  » https://doi.org/10.1017/s0007114520002688
• ³⁰
  Ares G, Vidal L, Allegue G, Giménez A, Bandeira E, Moratorio X, et al Consumers’ conceptualization of ultra-processed foods. Appetite. 2016;105:611-617. https://doi.org/10.1016/j.appet.2016.06.028
  » https://doi.org/10.1016/j.appet.2016.06.028
• ³¹
  Louzada ML, Baraldi LG, Steele EM, Martins AP, Canella DS, Moubarac J-C, et al Consumption of ultra-processed foods and obesity in Brazilian adolescents and adults. Am J Prev Med. 2015;81:9-15. https://doi.org/10.1016/j.ypmed.2015.07.018
  » https://doi.org/10.1016/j.ypmed.2015.07.018

Publication Dates

• Publication in this collection
  25 July 2022
• Date of issue
  2022

History

• Received
  27 Aug 2021
• Reviewed
  17 Dec 2021
• Accepted
  16 Mar 2022