Acessibilidade / Reportar erro

Citizen science and the (re)discovery of walkability from a child and youth perspective

Abstract

Encouragement of walking and the use of citizen science directly influence the promotion of positive aspects for society, especially for children and adolescents. The article presents a study conducted with students from a public school, aged between 10 and 12 years, that enhances a walkability assessment tool from the perspective of the younger population. Using a participatory methodology, the study integrates pedestrian perceptions as a determining factor in the analysis of the importance of indicators. The results showed that walkability is still inadequate in the studied region and that the inclusion of the public in the research process brought benefits that go beyond the quantitative weights assigned to the index scores.

active mobility; walkability; citizen science; participatory methodology; child and youth public

Resumo

O incentivo à caminhada e o emprego de uma ciência cidadã influenciam diretamente na promoção de aspectos positivos à sociedade, principalmente às crianças e aos adolescentes. Este artigo apresenta um estudo desenvolvido junto a estudantes de uma escola pública, com idades entre 10 e 12 anos, o qual aprimora uma ferramenta de avaliação da caminhabilidade sob a ótica do público infantojuvenil, integrando a percepção dos pedestres como fator determinante do cálculo da análise do grau de importância dos indicadores, através do uso de uma metodologia participativa. Os resultados mostraram que a caminhabilidade ainda é precária na região estudada e que a inclusão do público no processo de pesquisa trouxe benefícios que transcendem os pesos quantitativos atribuídos às notas do índice.

mobilidade ativa; caminhabilidade; ciência cidadã; metodologia participativa; público infantojuvenil

Introduction

Encouraging active mobility, especially walking, whether independently or in association with public transport, constitutes a favorable strategy to promote several benefits for society, related to urban road traffic flow, environmental and acoustic pollution, as well as public health (De Hartog et al., 2010DE HARTOG, J.; BOOGAARD, H.; NIJLAND, H.; HOEK, G. (2010). Do the health benefits of cycling outweigh the risks? Environmental Health Perspectives. Durham, v. 118, n. 8, pp. 1109-1116.). Furthermore, walking is widely considered an essential factor in fostering live communities, facilitating the interaction among neighborhoods and contributing to the creation of a more pleasant and safe urban environment to live in (Cambra, 2012CAMBRA, P. J. M. (2012). Pedestrian accessibility and attractiveness: indicators for walkability assessment. Dissertação de mestrado. Lisboa, Instituto Superior Técnico Lisboa.).

In the wake of this context, the need to design policies and strategies that prioritize pedestrian movement emerges, and the concept of "walkability" has proven to be of fundamental importance. This, from the English term walkability, can be defined as the quality of the walking environment or space intended for pedestrians (Park, 2008PARK, S. (2008). Defining, measuring, and evaluating path walkability, and testing its impacts on transit users' mode choice and walking distance to the station. Tese de doutorado. Berkeley, University of California.). The feasibility of measuring aspects, which influence walkability, therefore, presents itself as a means of deepening the understanding of both pedestrians and the surrounding environment, with emerging formulation of index as an increasingly widespread practice for understanding this phenomenon.

However, Moura, Cambra and Gonçalves (2017) emphasize that walking cannot be conceived as a uniform and universal entity. Thus, the built environment elements that affect the walking practice probably vary depending on factors such as: characteristics of pedestrians (age/gender/physical conditions); the purpose of walking (utility/leisure); the urban context and other environmental and cultural elements.

In this context, even though studies on active mobility are gaining constant relevance, and a variety of index have been proposed to measure walkability, few incorporate the unique perception of pedestrians in the calculations, especially related to the weighting of the relative importance of the indicators. Socially more vulnerable groups’ consideration, including children and young people, is an even scarcer practice. Such deficiencies may be interpreted as gaps in the analysis of the phenomenon, as they have a direct impact on the understanding of the attraction of the space for different population segments.

Considering the importance of walkability (re)thought from the perspective of children and young people and the potential for reconfiguring the environment based on their perception, a question arises about the proximity of the users themselves in the scientific context, as well as these subjects’ level of comprehension and engagement. Although there is the possibility for users to get involved in researches, predominantly in the data collection phase, the occasional and limited participation of the public can result in a distance between those subjects and the topic investigated. This compromises the ability not only to deepen the research of the given phenomenon, but also the ability to involve the population in the intrinsic relevance of the research.

It may be inserted here the discussion on the concept of "citizen science". Although its definition may vary depending on the authors, Parra (2015)PARRA, H. Z. M. (2015). "Ciência cidadã: modos de participação e ativismo informacional". In: ALBAGLI, S.; MACIEL, M. L.; ABDO, A. H. (eds.). Ciência aberta, questões abertas. Brasília, IBICT; Rio de Janeiro, Unirio. conceives it as a set of collaborations between professional scientists and interested citizens, focusing primarily on the involvement of population in general in scientific research activities. In this context, citizens actively contribute to science through their intellectual resources, local knowledge and available tools. In this panorama characterized by openness, a transdisciplinary approach and interconnection in networks, there is an improvement in interactions between the spheres of science, politics and society, moving towards the democratization of scientific research based on evidence (Socientize Consortium, 2013)SOCIENTIZE CONSORTIUM (2013). Green Paper on Citizen Science. European Commission. Disponível em: https://digital-strategy.ec.europa.eu/en/library/green-paper-citizen-science-europe-towards-society-empowered-citizens-and-enhanced-research. Acesso em: 25 jul 2021.
https://digital-strategy.ec.europa.eu/en...
. Consequently, citizen science presents itself as a potential means of co-creating solutions, adding practical citizen knowledge to the researcher's scientific knowledge.

Conclusively, given the relevance of the questions previously presented, the general objective of the article is to improve a walkability assessment tool from the perspective of children and young people, integrating the perception of pedestrians as a determining factor in calculating the analysis of the degree of importance of the indicators’ importance factor, using a partaking methodology.

Urban mobility, active mobility and walkability

Urban mobility plays an extremely important role in the lives of citizens, being intrinsically linked to their fundamental rights, as housing, health and leisure. However, despite having the capacity to contribute positively, urban mobility also plays a significant role in perpetuating socio-spatial disparities. This phenomenon is largely associated with the inadequate distribution of public space, which has been shaped throughout the time by the predominance of individual motorized transport (IDB and MDR, 2020).

In this context, Silva (2012)SILVA, A. J. P. A. (2012). Proposta de mobilidade urbana e ocupação territorial utilizando análise multicritério: estudo de caso aplicado ao Município de Viçosa e área de influência. Tese de doutorado. Viçosa, Universidade Federal de Viçosa. proposes that an approach capable of resolving this situation would be the adoption of a new paradigm, focusing on the principles of sustainable urban mobility and social inclusion. Therefore, the promotion of active mobility emerges as a valuable strategy to achieve this end.

The change in the legal framework towards this new paradigm has already begun, notably through the promulgation of law no. 12,587/2012, which establishes the National Urban Mobility Policy, which conceptualizes active mobility as modalities that adopt human effort or animal traction (Brasil, 2012BRASIL (2012). Lei n. 12.587, de 3 de janeiro. Institui as diretrizes da política nacional de mobilidade urbana. Disponível em: https://www.planalto.gov.br/ccivil_03/_ato2011-2014/2012/lei/l12587.htm. Acesso em: 11 mar 2023.
https://www.planalto.gov.br/ccivil_03/_a...
). This law makes clear the crucial relevance of active mobility for the success of the policy, aiming to value the act of walking as a clean, healthy and desirable city type of transportation. On this basis, it is clear that the concept of active mobility inherently encompasses the idea of walkability and the prioritization of walking on foot.

In order to deepen the walkability awareness, Battista and Manaugh (2018)BATTISTA, G. A.; MANAUGH, K. (2018). Stores and mores: toward socializing walkability. Journal of Transport Geography, v. 67, pp. 53-60. reinforce that built environment factors alone cannot change pedestrian behavior and equally important are their social characteristics. In this sense, it is important to understand the multiple aspects linked to walkability, using research approaches that allow a qualitative and quantitative analysis, like the use of index that are able to include social factors that shape the behavior of pedestrian routes.

Furthermore, as highlighted by Moura, Cambra and Gonçalves (2017), the perception of walking is not uniform among all groups of pedestrians, since the appreciation of elements related to this activity varies depending on people's perspectives. Given this reality, it is crucial to understand the particularities faced by children and young people, including their needs and capabilities when it comes to shaping new paradigms for urban mobility.

Urban spaces are often not adequately planned considering children's needs, resulting in a greater risk of traffic accidents and injuries when they walk, compared to other pedestrian groups (BID and MDR, 2020). Furthermore, Torres (2009) highlights that children have a unique perspective, with their own needs and aspirations, which leads them to perceive the environment differently than adults. Since the 1970s, several studies conducted in different countries have corroborated this notion, demonstrating that children are keen observers of their surroundings.

Horgan, Fernández and Kitching (2022) prove in their study that even children as young as 11-year-old ones know and understand their community, including its location, its risks and its values. This, in turn, demonstrates that children's local knowledge should be valued and sought out in school and community interventions to promote general well-being.

Finally, Horton et al. (2014)HORTON, J.; CHRISTENSEN, P.; KRAFTL, P.; HADFIELD-HILL, S. (2014). 'Walking… just walking': how children and young people's everyday pedestrian practices matter. Social & Cultural Geography, v. 15, n. 1, pp. 94-115. emphasize that the simple act of walking – an everyday activity and apparently without purpose – plays a central role in the lives, experiences and friendships of most children and young people. The learning process in and through the city, therefore, plays a vital role in child and youth development (Quintáns, 2015QUINTÁNS, I. (2015). "A importância do caminhar nos espaços urbanos para a educação, saúde e desenvolvimento da criança". In: DUARTE, T. L. O pedestre e a cidade. ANTP, Associação Nacional de Transportes Públicos. Série cadernos técnicos - Cidades a pé. Disponível em: http://www.antp.org.br/biblioteca-vitrine/cadernos-tecnicos.html. Acesso em: 7 maio 2023.
http://www.antp.org.br/biblioteca-vitrin...
). Under this approach, it is recognized that the city offers numerous educational opportunities, providing a variety of sensory stimuli, social interactions, historical and cultural contexts, as well as challenges that promote critical thinking and problem solving. Hence, Urban Pedagogy highlights the importance of offering educational experiences that integrate the city as a dynamic learning space, giving children and young people the opportunity to actively engage with the environment and become conscious and participating citizens.

Urban Pedagogy, citizen science and partaking methodologies

Urban Pedagogy is an educational area that focuses on interactions between the city and the learning process. According to Páramo (2009)PÁRAMO, P. (2009). Pedagogía Urbana: elementos para su delimitación como campo de conocimiento. Revista Colombiana de Educación. Bogotá, n. 57, pp. 14-27., when approaching Urban Pedagogy, it is crucial to consider not only the educational system, the school institution and educational communication, but also the pedagogical principles, framing the city as a formative context.

The interconnection between urban mobility and public (educational) space, with the stimulation of aspects related to citizenship, depends predominantly on a vital element: autonomy. In the matter of it, autonomy takes on two dimensions: the autonomy of the learner, including children and young people in their pedagogical journey (Freire, 1996FREIRE, P. (1996). Pedagogia da autonomia: saberes necessários à prática educativa. São Paulo, Paz e Terra.), and the ability of these individuals to enjoy public space through independent and improved mobility (GDCI, 2020GDCI - Global Designing Cities Initiative Rockefeller Philanthropy Advisors (2020). Designing Streets for Kids. Disponível em: https://globaldesigningcities.org/publication/designing-streets-for-kids/. Acesso em: 1º abr 2023.
https://globaldesigningcities.org/public...
). Both play a role in education and training active citizens, whether in formal or informal environments.

Learning, therefore, flourishes through social practice. From this perspective, the theory of situated learning, proposed by Lave and Wenger (1991)LAVE, J.; WENGER, E. (1991). Situated Learning: legitimate peripheral participation (Learning in doing: social, cognitive and computational perspectives). Cambridge, Cambridge University Press., challenges the traditional notion of learning as an isolated and individual process. Instead, it suggests that learning occurs when actively participating in communities of practice, in which learners engage in real-world activities, collaborate with other group members and develop a contextualized and situational understanding of pertinent knowledge and skills.

Integration of the learning process situated alongside discussions about autonomy, citizenship and Urban Pedagogy converges directly towards the creation of a new paradigm in the production of knowledge. This, in turn, is aligned with the notion of a citizen science that involves the generation of knowledge, meeting the needs and concerns of citizens, being developed and implemented by them (Ávila et al., 2022ÁVILA, R. de F.; TONIOLO, M. A.; MACIEL, L.; BRANCO, E. A. (2022). Avaliação de um processo de coprodução de conhecimento e engajamento de atores a partir de ferramentas da Educação Ambiental: práxis e ciência cidadã. Revista Brasileira De Educação Ambiental (RevBEA). São Paulo, v. 17, n. 3, pp. 371-391.). Consequently, the principles of citizen science associated with the democratization of access to knowledge and active participation in solving problems identified in the community's daily life.

The use of citizen science allows the population to contribute and learn simultaneously in the process not only about the problem presented here, but also about collective responsibility in the construction and knowledge dissemination. This scientific learning aimed at children and young people can cultivate the notion of citizenship, the ability to develop their own voices and the democratic participation that they will maintain as adults, influencing culture, behaviors and paradigms in the near future (Veloso, 2018VELOSO, M. T. (2018). Crianças na cidade - participação infantil no planejamento e gestão das cidades: novas espacialidades, autonomia, possibilidade. Dissertação de mestrado. Belo Horizonte, Universidade Federal de Minas Gerais.).

To understand the different approaches to citizen science and its impact on society, it is essential, firstly, to understand the different types of participatory methodologies and their influence on the engagement of the public involved. Streck (2016) highlights three substantial changes when comparing conventional scientific investigations with those that adopt participatory methods. First, in the conventional context, research often seems disconnected from the group it intends to study, with responding to questionnaires being perceived as an obligation. Secondly, traditional researchers often do not establish links that allow mutual engagement, which results in superficial knowledge about aspects of the community studied. Finally, research often does not leave a lasting impact on the life of the community, creating a distance between study and experience.

Using partaking methodologies, therefore, not only affects the study itself and the understanding of the role of the community, but also fosters community engagement in the topic. Furthermore, recognizing children and young people as active subjects, rather than objects of research, implies accepting that they have the ability to express themselves and report legitimate views and experiences. Green (2015)GREEN, C. (2015). Toward young children as active researchers: a critical review of the methodologies and methods in early childhood environmental education. The Journal of Environmental Education. Londres, v. 46, n. 4, pp. 207-229., when citing Barratt Hacking et al. (2013), reinforces that children must be involved as active researchers as they have the right to participate in issues relevant to them and, additionally, they offer unique perspectives to understand their own environments.

Methodology

This article’s methodology incorporates the fusion of quantitative and qualitative methods, characterized as a qualitative approach with quantitative treatment. The adoption of this hybrid approach finds its justification in the aspiration to unite knowledge of a specialized technical nature, predominantly based on quantitative treatment, with the popular knowledge of the student co-researchers invited to participate in the study, anchored in the use of qualitative tools.

The research was conducted from the perspective of citizen science, emphasizing citizen collaboration in scientific development. This aimed not only to achieve more assertive results on walkability, but also to promote an exchange of mutual learning.

The central basis of the study was to improve the walkability index produced for Belo Horizonte, based on the direct continuity of the works by Carvalho (2018)CARVALHO, I. R. V. (2018). Caminhabilidade como instrumento de mobilidade urbana: um estudo de caso em Belo Horizonte. Dissertação de mestrado. Belo Horizonte, Universidade Federal de Minas Gerais. and Barros (2021)BARROS, R. M. (2021). A infância e o pedestrianismo: um estudo exploratório da percepção de crianças sobre indicadores de caminhabilidade. Dissertação de mestrado. Belo Horizonte, Universidade Federal de Minas Gerais., since all of them, including the present study, are part of the same research project, representing a foreseen development. The first phase of the research developed a walkability index that makes it possible to assess the quality of the walking environment on a microscale, in accordance with the specificities of the capital of Minas Gerais. The second phase, based on this work, created an index to measure the friendliness of streets from the perspective of children. The present research, based on this, intended not only to review the evaluation and data collection method, but also to integrate the perception of pedestrians as a determining factor in the calculation of the analysis of the indicators’ importance level through a participatory methodology.

In this sense, the research was divided into five methodological stages: (1) application of the participatory method with co-researchers; (2) application of the technical analysis method on walkability; (3) walkability index score calculation (with weighting); (4) comparative analysis of results (with and without weighting) and (5) presentation of the results and collective construction of the action plan. The method was tested in a case study, based on a partnership with a public education institution, the Governador Carlos Lacerda Municipal School, located in Ipiranga neighborhood, in the city of Belo Horizonte (Minas Gerais). The institution offers Elementary and Middle School education, in addition to participating in the municipal Integrated School program, allowing students to carry out diverse personal, social, moral and cultural activities. The study was conducted in 2022 and 2023, involving a class with 16 students aged between 10 and 12 years old, presenting themselves in a period of transition between childhood and adolescence. In 2022, students were in their 5th grade and studied in the afternoon. In 2023, they were in 6th grade and started studying in the morning. All students were identified as co-researchers, given their active role in the research.

The initial stage of applying the participatory method with co-researchers was developed entirely at the partner school. This stage was conducted in six meetings, as presented in Chart 1.

Chart 1
– Co-researchers activities itinerary

A final meeting was held later, aimed at discussing the results and collectively constructing an action plan. A total number of seven meetings were held with co-researchers to formulate the study.

The research aimed, therefore, to encourage the participation of young pre- -adolescents beyond the role of target audience researched. In this sense, it was proposed that they occupy an active position in some definitions, for example in choosing their own fictitious names (pseudonyms to be used in the research) that would be written on a badge at all stages of the process, and in delimiting the area of application of the study, based on collective constructions in collaborative workshops. Once the co-researchers made set up these definitions, the process of systematizing the perception of children and young people regarding the walkability index indicators began – a method developed by Carvalho (2018)CARVALHO, I. R. V. (2018). Caminhabilidade como instrumento de mobilidade urbana: um estudo de caso em Belo Horizonte. Dissertação de mestrado. Belo Horizonte, Universidade Federal de Minas Gerais. and improved by Barros (2021)BARROS, R. M. (2021). A infância e o pedestrianismo: um estudo exploratório da percepção de crianças sobre indicadores de caminhabilidade. Dissertação de mestrado. Belo Horizonte, Universidade Federal de Minas Gerais..

The aim of the methodology was to capture the students' perception in relation to each indicator of the walkability index, taking into account potential differences when comparing them. The inclusion of perception in the index calculation arose due to the identification of a common gap in studies working with walkability index. Although pedestrian perception is often considered when selecting indicators, this consideration does not extend to the results’ calculation. This neglects the possibility of one indicator being more important than others, depending on individual perspectives, especially considering audiences such as children and adolescents who have particular needs.

Thus, to achieve the systematization of their perception, the combination of a series of objective and subjective methodological techniques was proposed: (1) development of focus groups, incorporating some typical elements of structured interviews; (2) development of playful dynamics using games and drawings to understand the individual and collective perception of participants; and (3) application of the Analytic Hierarchy Process (AHP) method. To achieve the objectives of the study and empower young people as co--research scientists, the methodology was adapted to be participatory, in an accessible language to everyone.

The focus groups were carefully planned in order to encourage each person's protagonism, using dynamics and resources with minimal researcher intervention. With this approach, after the first two meetings, a specific focus group was organized to debate the meaning of "active mobility", "walkability" and to understand each indicator and categories included in the index. During these meetings, the researcher asked the young people what each term meant to them, allowing meanings to be collectively constructed, without imposing a unilateral technical definition.

Based on the understanding of the walkability index, as well as all the indicators, both processes were conducted in the first and second meetings; the dynamics of systematization of perception were based on three sequential dynamics: (1) development of drawings that represented the school street (current situation and situation desired by the co-researchers); (2) application of the “Dream Street” dynamics associated with a memory game; and (3) application of paired comparisons, in accordance with the Analytical Hierarchical Process (AHP).

Initially, the co-researchers were asked, based on the concepts discussed during previous meetings, to create two drawings: one representing the school street as they currently perceive it and the other portraying how they would like the street to be. This playful activity aimed to focus on the most relevant elements, aligned with the individual perception of each participant.

Subsequently, a memory game was applied to continue the systematization of the co-researchers' perception. In the dynamics, the material made available by the Instituto Corrida Amiga on its website (Instituto Corrida Amiga, 2022) was used. The objective was to identify, in a relaxed way and through the game, the elements that would be significant for the collective perception of walkability on any street, going beyond the specific school context.

In an attempt to organize individual perception, an interpretative reading of the drawings was used. Each element presented in the representations of the streets was cataloged and subsequently grouped according to the most appropriate walkability index indicator. In a similar way, each element highlighted as desirable by the groups in the memory game was also cataloged and grouped according to the corresponding walkability index indicator. This process allowed us to develop an analysis of the most crucial elements for walkability, based on a compilation of the co-researchers individual and collective perception.

Based on the construction of the perception’s systematization, the relevance assessment of each indicator was developed in accordance with the criteria established for the multi-criteria weighting method, entitled Analytical Hierarchical Process (AHP). Originated by Thomas Saaty in the 1970s and structured in the book The Analytic Hierarchy Process: Planning, Priority Setting, Resource Allocation (Decision Making Series) by the same author in 1980, AHP is a multi-criteria decision method developed to assist in decision making whether for quantitative or qualitative factors.

To the method performed in this study, a simplified version of the AHP was applied, in which the objective was to obtain the weights of each indicator after the paired analysis of the criteria. It was defined that it would not have the structure for analyzing alternatives, as pointed out by Saaty in his methodology. The application, therefore, would be limited to the criteria and the objective that guided the paired comparisons was driven by the question: “Which indicator is most influential for walkability?”.

To define the criteria that would be analyzed, a selection was made of those considered most relevant, in accordance to the data tabulated in the individual and collective playful dynamics previously performed. To this end, the indicators that appeared most in both dynamics were scored and a statistical selection analysis was developed, also covering the upper median of the results, guaranteeing the 3rd quartile (Q3). This analysis was developed as the application of all 27 indicators was unfeasible, and as the paired comparison activity of many elements was not attractive to children and young people and the time required was unattainable for the case study.

Furthermore, for parity judgments, it was decided to use a reduced numerical scale, being 1/9, 1/5, 1, 5 and 9, with their respective qualitative values: much less important, less important, equally important, more important and much more important. The adaptation was based on the belief that adopting the number of comparisons originally proposed by the author could result in difficulty in understanding the analysis among young people and generate complexity that would result in a lack of interest in applying the method.

Since paired comparisons were made only with the criteria that were identified as the indicators resulting from the screening of the walkability index proposed by Carvalho (2018)CARVALHO, I. R. V. (2018). Caminhabilidade como instrumento de mobilidade urbana: um estudo de caso em Belo Horizonte. Dissertação de mestrado. Belo Horizonte, Universidade Federal de Minas Gerais., the matrix was scaled in accordance with this number. In the present study therefore, a 7x7 matrix was applied, representing 21 paired comparisons.

In an attempt to optimize time and encourage the attractiveness of paired comparisons dynamics, this was developed on a tablet application given to each co-researcher individually. This application was only viable since the educational institution where the case study was conducted had the equipment available for use.

Once all the values of the judgment matrix from the indicators selected in the screening were obtained, the normalized matrix was produced (divided by the priority of the criteria), considering the calculation of the numerical scale value of the parity judgment divided by the sum total of all numerical scale values of all parity judgments involving that particular criterion. This calculation was carried out for all selected indicators. Obtaining the normalized matrix, the weight value of each criterion was calculated using the simple arithmetic mean, through the total sum of all values in the normalized matrix for each criterion, divided by the number of criteria (in this specific case, 7). Finally, the weighting value of each indicator selected in the screening of those most relevant to co-researchers was obtained according to the individual perception of each one.

Following the partaking method, activities were developed in the application stage of the analysis on walkability technical method. This methodological phase involved collection and calculation of the walkability index developed for Belo Horizonte, capital of the state of Minas Gerais.

Once the spatial scope of the study was defined with the co-researchers, it became possible to collect data for each indicator that compose the walkability index. The process was developed in accordance with the method recommended by Carvalho (2018)CARVALHO, I. R. V. (2018). Caminhabilidade como instrumento de mobilidade urbana: um estudo de caso em Belo Horizonte. Dissertação de mestrado. Belo Horizonte, Universidade Federal de Minas Gerais., considering the discussions and developments made by Barros (2021)BARROS, R. M. (2021). A infância e o pedestrianismo: um estudo exploratório da percepção de crianças sobre indicadores de caminhabilidade. Dissertação de mestrado. Belo Horizonte, Universidade Federal de Minas Gerais., guiding the 27 indicators that make up the 8 categories of the index. Note that the index used was developed considering the specific characteristics of the capital of Minas Gerais, turning it essential to evaluate the context to be studied, in case there is any intentions to replicate the method. The categories analyzed and their indicators are detailed below:

  • Accessibility: effective sidewalk width, conservation of the sidewalk pavement, unevenness in the sidewalk, tactile paving and topography;

  • Attractiveness: visual attractiveness, visual permeability, afforestation, cleanliness and noise pollution;

  • Connectivity: block measure, access to public transport and cycling infrastructure;

  • Comfort: benches to sit on, protection against adverse weather conditions and risk of flooding;

  • Public safety: presence of pedestrians (security) and lighting;

  • Road safety: possibility of conflict between pedestrians and vehicles on the sidewalk, protection barriers (buffers), width of the carriageway and vehicle speed;

  • Land use: mixed land use and parks and green areas within walking distance;

  • Crossing: access to the crossing, number of streets at the intersection and traffic signs at the intersection.

The unit of analysis for the walkability index is the sidewalk segment, which can be defined as the “part of the street located between adjacent intersections of the pedestrian network – including non-motorized intersections –, taking into account only one side of the sidewalk” (ITDP Brazil, 2018, p. 17). Data collection for each indicator was performed in loco or investigated through the use of platforms and/or auxiliary documents, like Google Maps.

In order to calculate the walkability index score, it is necessary to normalize the results investigated to adjust them to the same variation pattern starting from the collection and tabulation of data for each indicator, so that the results are comparable. For this reason, the methodology proposed by Eastman and Jiang (1996)EASTMAN, J. R.; JIANG, H. (1996). Fuzzy Measures in Multicriteria Evaluation. In: PROCEEDINGS, SECOND INTERNATIONAL SYMPOSIUM ON SPATIAL ACCURACY ASSESSMENT IN NATURAL RESOURCES ENVIRONMENTAL STUDIES. Fort Collins, Colorado, pp. 527-534. was adopted.

Equation 1: Normalization of indicator scores calculation

X i = ( R i R min ) ( R mix R min ) × I N

In which:

Xi = normalized value;

Ri = value of the variable to be normalized;

Rmín = minimum value of the variable;

Rmáx = maximum value of the variable;

IN = normalized range.

The result of the categories is obtained through the simple arithmetic mean of the indicators’ results that compose it.

Equation 2: Calculation of the score for the walkability index categories

N C = A n n

In which:

NC = final grade for the category;

∑An = sum of the score of the category indicators;

n = number of category indicators.

In the end, to obtain the final walkability index score for each segment, the simple arithmetic mean of all categories is calculated.

Equation 3: Calculation of the walkability index score for each segment

N I C = C n n

In which:

NIC = final walkability index score;

∑An = sum of the category scores;

n = number of categories.

It should be noted that the score calculated for each indicator, category or index also presents a qualitative variation according to the range of its results, as proposed by Barros (2021)BARROS, R. M. (2021). A infância e o pedestrianismo: um estudo exploratório da percepção de crianças sobre indicadores de caminhabilidade. Dissertação de mestrado. Belo Horizonte, Universidade Federal de Minas Gerais.: terrible (from 0 to 20.99); bad (from 21 to 40.99); regular (from 41 to 60.99); good (from 61 to 80.99) and excellent (from 81 to 100).

Through the calculation of the systematization of students' perception, based on the adaptation of the AHP multi-criteria weighting method, the weights of each indicator have already been calculated through the simple arithmetic mean of the normalized matrix. These, therefore, already reflect the degree of relevance of each indicator recognized as most important, according to the individual perspective of young people's perception.

In this scenatio, to achieve collective weighting of the indicators, the Mahalanobis statistical distance method was applied, as an attempt to systematize the perception of the group of co-researchers involved in the research. Introduced by Indian mathematician Prasanta Chandra Mahalanobis, the method is based on correlations among variables with which distinct patterns can be identified and analyzed (Mahalanobis, 1936). Hair et al. (2009HAIR, J. F.; BLACK, W. C.; BABIN, B. J.; ANDERSON, R. E.; TATHAM, R. L. (2009). Análise multivariada de dados. Porto Alegre, Bookman., p. 78) define the method as “a multivariate assessment of each observation across a set of variables” responsible for measuring “the distance of each observation in a multidimensional space from the average center of all observations, providing a single value for each observation, regardless of the number of variables in question” (ibid.).

Based on this, to obtain the weighted walkability index score, each indicator had a weight equal to 1 and the indicators that were selected as the most relevant in the screening process of the systematization of perception had a weight equal to 1 + x, x being the value obtained in the Mahalanobis distance calculation. The multiplication factor therefore was applied to each already standardized indicator score, obtained using the walkability index method proposed by Carvalho (2018)CARVALHO, I. R. V. (2018). Caminhabilidade como instrumento de mobilidade urbana: um estudo de caso em Belo Horizonte. Dissertação de mestrado. Belo Horizonte, Universidade Federal de Minas Gerais. and adapted by Barros (2021)BARROS, R. M. (2021). A infância e o pedestrianismo: um estudo exploratório da percepção de crianças sobre indicadores de caminhabilidade. Dissertação de mestrado. Belo Horizonte, Universidade Federal de Minas Gerais.. Only then was the simple mean of the indicators for the category scores and the simple mean for the overall score of the walkability index for each segment carried out, as already predicted by Carvalho (2018)CARVALHO, I. R. V. (2018). Caminhabilidade como instrumento de mobilidade urbana: um estudo de caso em Belo Horizonte. Dissertação de mestrado. Belo Horizonte, Universidade Federal de Minas Gerais..

Finally, after applying the participatory method of systematizing perception and the technical method of applying the walkability index, the students gathered for a final focus group, in which the research results were presented and discussed. To facilitate that everyone would understand, the results of the weighted walkability index were presented according to their qualitative assortment.

After the results’ presentation and discussion, a collective construction of an action plan was taken, prioritizing improvements that directly interfere with the worst indicators measured in the index. To mediate the process, methods were used that encouraged the participation and autonomy of the public involved. In this regard, the proposal was made for those involved to suggest ideas and solutions that were written on post-its and pasted above the name of the indicator investigated. After defining the action plan, as the last stage of closing the method, a conversation was proposed with the co-researchers about the experience of participating in the research, based on a survey of suggestions, criticisms and other raised contributions.

Application and research results

Regarding the results of the playful dynamics, two main ways of evaluating perception were used. One of them focused on the individual perception of each co-researcher (through the execution of drawings), and the other considered the collective perception (through the execution of the memory game). In regard to the drawing process, some young people have not felt comfortable drawing and wrote a text to highlight their ideas, explaining them to the researcher who was mediating the practice. Regarding the memory game, there was initial difficulty for some young people in forming a group. However, once the game started, they became interested in the dynamics again, requesting that the game could last longer than the planned time, which was not viable given the schedule of the classes.An interpretative reading of the results was developed based on the games played, through the tabulation and characterization of the elements, listing and framing them in the walkability index indicators. The interpretation method was promoted as an objective way of identifying the most relevant elements to co-researchers, in order to enable the AHP multi-criteria analysis method to be carried out afterward.

Through scoring, therefore, it was possible to screen the most relevant indicators for the group of co-researchers involved in the research. This process was implemented through a statistical analysis of sample selection to guarantee the upper quartile, or 3rd quartile (Q3), as shown in Table 1 and Figure 2. Recurrence refers to whether the indicator appeared in more than one drawing and the total refers to the number of drawings in which each indicator was mentioned. It should be noted that, for the selection of indicators using the statistical method, the fact that it was recurrent, in itself, did not influence the result, but rather the total number of drawings in which each indicator was presented.

Figure 1
– Playful dynamics of individual perception (left) and collective perception (right)

Table 1
– Screening the most relevant indicators

Figure 2
– Graphical representation of the statistical screening selection of indicators

By screening the most relevant indicators using the participatory methods conducted, it was possible to apply the adapted AHP multi--criteria analysis method. Paired comparisons were made individually with each co-researche, using a form on a tablet. Starting from the weights achieved by the individual approach, to achieve collective weighting of the indicators, the Mahalanobis statistical distance method was applied, as an attempt to systematize the perception of the group of co-researchers involved in the research. Summaries of the results of the statistical application are presented in Chart 2.

Chart 2
– Result of the weights by the Mahalanobis distance arrangement

With regard to the influence of using a participatory methodology on the involvement of children and young people in the topic addressed, it was found that, in addition to the used techniques, the application of the participatory method needs to be widely analyzed, so that participation can be stimulated indeed. Depending on the method approached for each dynamic, it is necessary to understand more about how the relationship between the young person's body and the proposed activity will occur, which can be (de)motivating for their involvement. An example experienced on this issue was the idea of the conversation circle for the focus group conducted, since the fact that the participants spent around 50 minutes sitting on the floor caused their dispersion on the subject to increase. Another example was verified by the young people's request to move around the place studied, which demonstrated that they not only wanted to study the concept of walkability, but to attest to their knowledge through the practice of walking in urban space.

The position of the co-researchers was also a relevant factor in verifying their involvement with the partaking method. The arrangement of tables and chairs in a circle for developing the drawings was motivating for the co-researchers to engage with each other, even though it was an individual dynamic, given that the furniture arrangement did not follow a hierarchical structure of the students' position.

Finally, another important aspect is related to autonomy. This was notice, mainly in defining the names to be adopted in the research freely and in the use of the badge in the meetings, which was essential to encourage their involvement.

In relation to the methodological approach called technique, the calculation, analysis and comparison of the walkability index scores with and without weighting were provided. Based on the results of the indicators and categories it is presented below the qualitative results of the final score of the walkability index, shown on the map of the studied area (Figure 3), as well as the comparative table between the index with and without weighting, with the quantitative and qualitative data obtained (Chart 3).

Figure 3
– Spatialization of the walkability index qualitative results

Chart 3
– Comparative results of the walkability index

There was a small difference in the results of the index with and without weighting, and the only segment that presented a qualitative difference in the grade classification is S2. Given this, two representations of spatialization of qualitative results were presented for a better comparative visualization.

With reference to the results, both from the weighted and non-weighted indexes, it appears that the area, for the most part, was classified as “bad”. The only segments that had a score considered “regular” were S1, S12 and S15, for the non-weighted index and, in addition to these, S2, for the weighted index. This demonstrates that walkability is still very precarious in the region and that there is great potential to improve aspects reported in each category, therefore, improving the final evaluation of the index.

After the implementation of the weighted walkability index, developed based on the application of the participatory method and other contributions added to the application and analysis of the in loco technical method, a final meeting was held with the co-researchers to reflect and discuss the process, the results and collectively create an action plan. This last stage was developed as a way to materialize the entire process and build strategies to expand the results to different segments of society, since the study aimed to be produced in light of citizen science.

During the focus group, the results of each indicator scored as “terrible” were discussed in a peaceful way, with all proposals for improvements recorded on post-its pasted on the indicator cards. The meeting provided an incredibly rich exchange of knowledge, and it was impressive to see the critical vision aligned with the creativity of the co-researchers when proposing solutions to problems. Starting from all the scored solutions, it was possible to tabulate them according to each indicator analyzed, making it possible to observe the final result in Chart 4.

Chart 4
– Results of improvements proposed in the collective action plan

It seems that, according to the result, numerous findings can be interpreted about the solutions scored, some of which are presented below:

  • The co-researchers presented a critical view on unused places and how to reverse this situation, mainly in the solutions of “making parks in empty spaces” and “making benches on top of the school”, the latter representing a place without a defined use but where the students see great potential;

  • Co-researchers not only identified more comprehensive solutions to problems, but also objective solutions aimed at specific spaces, as seen in the solutions “making a park at the school’s doorstep”; “build a bench in front of the Cultural Plant”; “make benches on top of the school” and “include benches in front of the school entrance”. These could be even more easily made in loco;

  • The co-researchers presented suggestions which are are directly related to their experiences of being in other places/situations, and it was verified that they view the aspects experienced as potential solutions to the problems identified. This demonstrated not only a critical look at their experiences, but creativity in their correlation with the problems identified. This was observed, mainly, in the propositions “make parklets”, “make a cable car”, “make an escalator”; “close the streets to cars”; “make trails with trees”; “create lakes” and “create windings”.

Final considerations

Urban mobility is an extremely complex topic, requiring an integrated approach in the context of urban planning and management. The promotion of a new model of urban mobility becomes imperative, encompassing both social and environmental concerns, and aiming to address the diverse demands of society. It is of special relevance to re-examine urban mobility from the perspective of children and young people, enabling them to have autonomy in their movements and effective participation as citizens in urban life.

In a paradigmatic change in the current context of urban mobility, which is still very dependent on the use of individual motor vehicles, the relevance of active mobility stands out, such as the act of walking, which is especially beneficial for children and young people. In addition to being a type of physical exercise that contributes to general health and well-being, active mobility has a positive influence on the motor development, coordination and spatial awareness of children and young people (Teles, 2019TELES, L. A. C. (2019). Os caminhos das crianças: a mobilidade ativa no trajeto escolar. Dissertação de mestrado. Brasília, Universidade de Brasília.).

Furthermore, engagement together with reflections on autonomy, citizenship and Urban Pedagogy directly contributes to the construction of a new paradigm of knowledge production. This new paradigm aligns with the concept of citizen science, which differs from technocratic scientific production by involving the creation of knowledge that meets the needs and concerns of citizens, while being developed and put into practice by them (Ávila et al., 2022ÁVILA, R. de F.; TONIOLO, M. A.; MACIEL, L.; BRANCO, E. A. (2022). Avaliação de um processo de coprodução de conhecimento e engajamento de atores a partir de ferramentas da Educação Ambiental: práxis e ciência cidadã. Revista Brasileira De Educação Ambiental (RevBEA). São Paulo, v. 17, n. 3, pp. 371-391.). Thus, the foundations of citizen science are intrinsically associated with the search for democratization of access to knowledge and active engagement in solving problems identified in the daily life of a community.

Taking into account the importance of active mobility and the relevance of adopting an approach based on the principles of citizen science, the general objective of the research was to improve a walkability assessment tool from the perspective of children and young people, based on the developments of walkability indices already produced for the city of Belo Horizonte by Carvalho (2018)CARVALHO, I. R. V. (2018). Caminhabilidade como instrumento de mobilidade urbana: um estudo de caso em Belo Horizonte. Dissertação de mestrado. Belo Horizonte, Universidade Federal de Minas Gerais. and Barros (2021)BARROS, R. M. (2021). A infância e o pedestrianismo: um estudo exploratório da percepção de crianças sobre indicadores de caminhabilidade. Dissertação de mestrado. Belo Horizonte, Universidade Federal de Minas Gerais.. This integrates the perception of pedestrians as a determining factor in the calculation of the analysis of the degree of importance of the indicators, through the use of a participatory methodology.

Based on the method application, with regard to the influence of the participatory methodology on the engagement of children and young people with the topic addressed, it can be concluded that, in a subjective analysis carried out on all meetings, there was, in fact, a significant engagement by part of the co- -researchers on the topic. This conclusion was based on observing their excitement during some activities, on the positive feedback received in the final meetings and, mainly, on demonstrating the knowledge acquired on the subject by proposing improvements for the collective construction of the action plan.

About the analysis of differences in results between the method that incorporates the perception of pedestrians as a determining factor in the weighting of walkability index indicators, in comparison with methodologies that disregard such aspects, it existis – one of the central objectives of the study conducted. It is clear that, although the difference is subtle, it exists. From this, it was concluded that the creation of a weighted method was relevant for the interpretation of the results, since the public's perspective influenced the final metric.

Furthermore, despite the modest discrepancy in the results, the inclusion of the public in the research process brought benefits beyond the quantitative weights attributed to the index scores. The process of understanding pedestrians' perception was fundamental for understanding other values, through interactions with co-researchers, providing an environment for exchanging learning on the topic for everyone involved in the research, being a transformative experience. It is noteworthy, therefore, that as important as the final result was the "walk", understood as valuable in all the senses attributed to the word, both in the sense of investigating walkability and the construction of environments more favorable to pedestrians, and in the sense to highlight the grandeur of the process and all the experiences attributed to it.

Finally, it concludes with the message of highlighting the importance of developing and applying studies that can contribute to a paradigm shift in society. This is relevant in encouraging walking, and all the positive characteristics intrinsic to this practice, in the production of citizen science, considering its assertiveness and impacts on population engagement, as well as the participation of children and young people in understanding structural and in building better cities.

Acknowledgements

The authors would like to thank the educators, teachers and directors of the Governador Carlos Lacerda Municipal School for making the partnership possible and for the commitment they put into the study, the SOnhANDO A PÉ group, for enabling contact with the educational institution, and the co-researchers students, whose commitment, dedication and engagement were fundamental to enable this work

Referências

  • ÁVILA, R. de F.; TONIOLO, M. A.; MACIEL, L.; BRANCO, E. A. (2022). Avaliação de um processo de coprodução de conhecimento e engajamento de atores a partir de ferramentas da Educação Ambiental: práxis e ciência cidadã. Revista Brasileira De Educação Ambiental (RevBEA). São Paulo, v. 17, n. 3, pp. 371-391.
  • BID - Banco Interamericano de Desenvolvimento; MDR - Ministério do Desenvolvimento Regional (2020). Disponível em: https://www.gov.br/mdr/pt-br/assuntos/mobilidade-e-servicos-urbanos/CTR_Mobilidadeap_compressed.pdf Acesso em: 12 mar 2023.
    » https://www.gov.br/mdr/pt-br/assuntos/mobilidade-e-servicos-urbanos/CTR_Mobilidadeap_compressed.pdf
  • BARROS, R. M. (2021). A infância e o pedestrianismo: um estudo exploratório da percepção de crianças sobre indicadores de caminhabilidade. Dissertação de mestrado. Belo Horizonte, Universidade Federal de Minas Gerais.
  • BATTISTA, G. A.; MANAUGH, K. (2018). Stores and mores: toward socializing walkability. Journal of Transport Geography, v. 67, pp. 53-60.
  • BRASIL (2012). Lei n. 12.587, de 3 de janeiro. Institui as diretrizes da política nacional de mobilidade urbana. Disponível em: https://www.planalto.gov.br/ccivil_03/_ato2011-2014/2012/lei/l12587.htm Acesso em: 11 mar 2023.
    » https://www.planalto.gov.br/ccivil_03/_ato2011-2014/2012/lei/l12587.htm
  • CAMBRA, P. J. M. (2012). Pedestrian accessibility and attractiveness: indicators for walkability assessment Dissertação de mestrado. Lisboa, Instituto Superior Técnico Lisboa.
  • CARVALHO, I. R. V. (2018). Caminhabilidade como instrumento de mobilidade urbana: um estudo de caso em Belo Horizonte. Dissertação de mestrado. Belo Horizonte, Universidade Federal de Minas Gerais.
  • DE HARTOG, J.; BOOGAARD, H.; NIJLAND, H.; HOEK, G. (2010). Do the health benefits of cycling outweigh the risks? Environmental Health Perspectives Durham, v. 118, n. 8, pp. 1109-1116.
  • EASTMAN, J. R.; JIANG, H. (1996). Fuzzy Measures in Multicriteria Evaluation. In: PROCEEDINGS, SECOND INTERNATIONAL SYMPOSIUM ON SPATIAL ACCURACY ASSESSMENT IN NATURAL RESOURCES ENVIRONMENTAL STUDIES. Fort Collins, Colorado, pp. 527-534.
  • FREIRE, P. (1996). Pedagogia da autonomia: saberes necessários à prática educativa São Paulo, Paz e Terra.
  • GDCI - Global Designing Cities Initiative Rockefeller Philanthropy Advisors (2020). Designing Streets for Kids. Disponível em: https://globaldesigningcities.org/publication/designing-streets-for-kids/ Acesso em: 1º abr 2023.
    » https://globaldesigningcities.org/publication/designing-streets-for-kids/
  • GREEN, C. (2015). Toward young children as active researchers: a critical review of the methodologies and methods in early childhood environmental education. The Journal of Environmental Education Londres, v. 46, n. 4, pp. 207-229.
  • HAIR, J. F.; BLACK, W. C.; BABIN, B. J.; ANDERSON, R. E.; TATHAM, R. L. (2009). Análise multivariada de dados Porto Alegre, Bookman.
  • HORGAN, D.; FERNÁNDEZ, E.; KITCHING, K. (2022). Walking and talking with girls in their urban environments: A methodological meandering. Irish Journal of Sociology, v. 31, n. 1, pp. 101-124.
  • HORTON, J.; CHRISTENSEN, P.; KRAFTL, P.; HADFIELD-HILL, S. (2014). 'Walking… just walking': how children and young people's everyday pedestrian practices matter. Social & Cultural Geography, v. 15, n. 1, pp. 94-115.
  • INSTITUTO CORRIDA AMIGA (2022). Materiais de uso livre. Disponível em: http://corridaamiga.org/materiais/ Acesso em: 6 nov 2022.
    » http://corridaamiga.org/materiais/
  • ITDP BRASIL - Instituto de Políticas de Transporte & Desenvolvimento (2018). Publicação Índice de Caminhabilidade 2.0: Ferramenta. Disponível em: https://itdpbrasil.org/icam2/ Acesso em: 29 out 2022.
    » https://itdpbrasil.org/icam2/
  • LAVE, J.; WENGER, E. (1991). Situated Learning: legitimate peripheral participation (Learning in doing: social, cognitive and computational perspectives) Cambridge, Cambridge University Press.
  • MAHALANOBIS, P. C. (1936). On the generalised distance in statistics. Proceedings of the National Institute of Sciences of India Allahabad, v. 2, n. 1, pp. 49-55.
  • MOURA, F.; CAMBRA, P.; GONÇALVES, A. B. (2017). Measuring walkability for distinct pedestrian groups with a participatory assessment method: a case study in Lisbon. Journal Landscape and Urban Planning Amsterdam, v. 157, pp. 282-296.
  • PÁRAMO, P. (2009). Pedagogía Urbana: elementos para su delimitación como campo de conocimiento. Revista Colombiana de Educación. Bogotá, n. 57, pp. 14-27.
  • PARRA, H. Z. M. (2015). "Ciência cidadã: modos de participação e ativismo informacional". In: ALBAGLI, S.; MACIEL, M. L.; ABDO, A. H. (eds.). Ciência aberta, questões abertas. Brasília, IBICT; Rio de Janeiro, Unirio.
  • PARK, S. (2008). Defining, measuring, and evaluating path walkability, and testing its impacts on transit users' mode choice and walking distance to the station Tese de doutorado. Berkeley, University of California.
  • QUINTÁNS, I. (2015). "A importância do caminhar nos espaços urbanos para a educação, saúde e desenvolvimento da criança". In: DUARTE, T. L. O pedestre e a cidade. ANTP, Associação Nacional de Transportes Públicos. Série cadernos técnicos - Cidades a pé. Disponível em: http://www.antp.org.br/biblioteca-vitrine/cadernos-tecnicos.html Acesso em: 7 maio 2023.
    » http://www.antp.org.br/biblioteca-vitrine/cadernos-tecnicos.html
  • SAATY, T. L. (1980). The analytic hierarchy process: planning, priority setting, resource allocation (Decision making series) Nova York, McGraw-Hill.
  • SILVA, A. J. P. A. (2012). Proposta de mobilidade urbana e ocupação territorial utilizando análise multicritério: estudo de caso aplicado ao Município de Viçosa e área de influência. Tese de doutorado. Viçosa, Universidade Federal de Viçosa.
  • SOCIENTIZE CONSORTIUM (2013). Green Paper on Citizen Science. European Commission. Disponível em: https://digital-strategy.ec.europa.eu/en/library/green-paper-citizen-science-europe-towards-society-empowered-citizens-and-enhanced-research Acesso em: 25 jul 2021.
    » https://digital-strategy.ec.europa.eu/en/library/green-paper-citizen-science-europe-towards-society-empowered-citizens-and-enhanced-research
  • STRECK,' D. R. (2016). Metodologias participativas de pesquisa e educação popular: reflexões sobre critérios de qualidade. Interface - Comunicação, Saúde, Educação Botucatu, v. 20, n. 58, pp. 537-547.
  • TELES, L. A. C. (2019). Os caminhos das crianças: a mobilidade ativa no trajeto escolar. Dissertação de mestrado. Brasília, Universidade de Brasília.
  • TORRES, J. (2009). Children & cities: planning to grow together. Ottawa, Vanier Institute of the Family.
  • VELOSO, M. T. (2018). Crianças na cidade - participação infantil no planejamento e gestão das cidades: novas espacialidades, autonomia, possibilidade. Dissertação de mestrado. Belo Horizonte, Universidade Federal de Minas Gerais.

Publication Dates

  • Publication in this collection
    13 May 2024
  • Date of issue
    May-Aug 2024

History

  • Received
    15 Aug 2023
  • Accepted
    2 Oct 2023
Pontifícia Universidade Católica de São Paulo Rua Ministro de Godói, 969 - 4° andar - sala 4E20 - Perdizes, 05015-001 - São Paulo - SP - Brasil , Telefone: (55-11) 94148.9100 - São Paulo - SP - Brazil
E-mail: cadernosmetropole@outlook.com