Type
|
External evaluation |
Methodology
|
An external evaluator observes the spaces and environments, identifies, and assigns quality to indicators. |
Indicators
|
ITERS-R Evaluates the space and furniture; personal care routines; speak and understand; activities; interaction; structure of the program, parents, and staff, with 39 items detailed in 455 indicators. |
ECERS-R Evaluates the space and furniture; personal care, language, and reasoning routines; activities; interaction, program structure; parents and staff, with 43 items and 470 indicators. |
Time
|
One day, 3.5-hour observation period. |
Participation
|
Do not feature participatory characteristic, only queries parents and team when the external evaluator cannot identify the indicators in the environment. |
Results
|
Data and information are collected and measured, and a value is attributed by the evaluator who does not discuss the results of the evaluation. |
Post-results
|
The external evaluator provides indications on points for improvement and action strategies. |
Viability
|
Viable for ease of access and little time in the preparation of external evaluators. |
Validity and reliability
|
Valid and reliable for being comprehensive with common parameters of early childhood education, facilitating quality mapping in diverse social and educational realities. |
Maturity
|
Degree of maturity established by having been applied in different historical times and in different contexts of the United States and later in countries of the five continents. |
Source: Harms et al. (2006Harms, T., Cryer, D., & Clifford, R. M. (2006). Infant/toddler environment rating scale. Teachers College Press.), Harms (2013) and Pimenta (2017Pimenta, C. O. (2017). Avaliações municipais da educação infantil: Contribuições para a garantia do direito à educação das crianças brasileiras? [Tese de doutorado, Universidade de São Paulo - Faculdade de Educação].). |
ISQUEN and AVSI - Italy |
Type
|
Meta-evaluation and self-assessment |
Methodology
|
Comprising an external evaluator, internal evaluators (among them, an articulator) and other segments of the institution (families and professionals). Present pre-defined indicators and promote debate among participants in a procedural and formative manner. |
Indicators
|
ISQUEN Subjects, contexts and practices, the knowledge of doing, guarantees. |
AVSI Educational experience, professional activities, adults, and their relations, guarantees, structure. |
Time
|
Varies between institutions. It does not prescribe the application time or frequency. |
Participation
|
Of a negotiated and dialog-based nature, provide for the participation of all segments. |
Results
|
Items in disagreement are socialized in order to reflect and find consensus from conception, values of that reality and the principles of early childhood education. |
Post-results
|
Drafting a collective action plan, involving everyone in improvements. |
Viability
|
Due to being procedural and given the interaction of the external evaluator, as articulator of the instrument, require higher qualification and more time. |
Validity and reliability
|
Valid and reliable. Feature internal consistency, the evaluation was constructed from the adaptation of ITERS and ECERS, foresee adaptation to contextual reality. |
Maturity
|
Degree of maturity established, also applied in other countries. However, due to evaluating the context, require adaptation to the reality observed. |
Source: Becchi et al. (2014Becchi, E., Bondioli, A., & Ferrari, M. (2014). ISQUEN: Indicadores e escala de avaliação da qualidade educativa da creche. In L. Cipollone (Org.), Instrumentos e indicadores para avaliar a creche: Um percurso de análise da qualidade (pp. 149-197). UFPR.), Pimenta (2017Pimenta, C. O. (2017). Avaliações municipais da educação infantil: Contribuições para a garantia do direito à educação das crianças brasileiras? [Tese de doutorado, Universidade de São Paulo - Faculdade de Educação].) and Souza et al. (2017Souza, G. de, Moro, C., França, F. F., & Rodrigues, A. J. L. (2017). A pesquisa em rede na educação infantil: Avaliação de contexto, modos de proceder e possibilidades de reflexão. RELAdEI: Revista Latinoamericana de Educación Infantil - Evaluación de Contextos en Educación Infantil, 6(1-2), 23-32.). |
ECCP - Mexico |
Type
|
External evaluation |
Methodology
|
Evaluation of programs carried out by external evaluators. The instrument combines observation of spaces and interviews with principals and faculty with respect to the institution and classrooms. |
Indicators
|
Establishment, resources, educational process, educational management, family and community relations, classroom, resources. |
Time
|
External evaluators: visits at the beginning and end of the school year. Two days with interviews, families, and children. Three visits: one to discuss the results, verifying their validity and two to develop improvement actions with teachers and principals. And, finally, three visits to check the deployment of said actions. |
Participation
|
The instrument combines observation and interviews, but the quality opinion is issued by external evaluators. |
Results
|
Improvements were noted in the quality of infrastructure, materials, management, health, institutional identity, daycare-family relationship. |
Post-results
|
Evaluation feedback is the basis for the continuity of curricula and reorienting teacher training and supervision. |
Viability
|
Instrument constructed by the Mexican educational system between 2003 and 2007, sought to adapt to the guidelines (general principles of early childhood education), viable for that context. |
Validity and Reliability
|
Not described. Instrument in test period for a standard evaluation for a diverse and multicultural context, which coincided with the expansion of the offer of openings. |
Maturity
|
Maturity level not defined. Has been reformulated five times, including new dimensions: health, climate, children with disabilities. |
Source: Myers (2011Myers, R. (2011). Em busca da qualidade educacional na pré-escola: Uma experiência mexicana. Cadernos de Pesquisa, 41(142), 100-115.) and Martínez Preciado (2010Martínez Preciado, J. F. (2010). La construcción de indicadores y evaluación de la calidad en centros educativos: Seis experiencias en México. REICE: Revista Iberoamericana sobre Calidad, Eficacia y Cambio en Educación, 8(5), 133-153.). |
NQS - Australia |
Type
|
External evaluation |
Methodology
|
External evaluator evaluates the quality of services. Visits of authorized agent to observe the spaces, question, analyze documents, check the application of the plan. |
Indicators
|
Educational and practical program; children’s health and safety; physical environment; personnel arrangements; relationships with children; collaborative partnerships with families and communities; leadership and management of services. |
Time
|
Application time in not described. |
Participation
|
Participation limited to professionals and families. External evaluators have major influence on the decision on the outcome. |
Results
|
Legislation provides for the dissemination of results (regulatory assessment). Quality levels must be achieved for accreditation of institutions. |
Post-results
|
Penalties are provided by law for institutions not meeting quality levels established in the improvement plan. Subject to accountability of professionals, institutions, and suspensions of services. |
Viability
|
Viable. Because it is established, articulated, and guaranteed by law, resources for execution are provided for. |
Validity and reliability
|
Recognizes the existence of multicultural contexts, presents coherence with national guidelines, making the instrument reliable and valid, but sets precedents for ranking of institutions. |
Maturity
|
It has consistent maturity level. It is systemic, comprehensive, nationally standardized. |
Source: Tayler (2014Tayler, C. (2014). Avaliação da qualidade da educação infantil na Austrália. Estudos em Avaliação Educacional, 25(58), 126-151.) and Pimenta (2017Pimenta, C. O. (2017). Avaliações municipais da educação infantil: Contribuições para a garantia do direito à educação das crianças brasileiras? [Tese de doutorado, Universidade de São Paulo - Faculdade de Educação].). |
IDEA - Spain |
Type
|
Self-assessment |
Methodology
|
Voluntary participation by institution. Comprised of steps: a) information collection: interviews, review of documents, observation, application of questionnaires; b) valuing: drafting of reports and highlighting strengths and weaknesses; c) decision-making: proposals for improvement. |
Indicators
|
Faculty qualification, number of children per classroom, physical structure in relation to health and safety (in buildings, facilities, indoor and outdoor equipment), the curriculum and learning and educational practice. |
Time
|
Instrument deployment time is not described. |
Participation
|
Participation conditional on the evaluation questionnaire applied to families and faculty. External evaluators define the level of quality. |
Results
|
Highlights institution’s weaknesses and strengths. |
Post-results
|
Improvements are proposed, but no description on how to do it. |
Viability
|
Does not describe difficulty or success in the application. |
Validity and reliability
|
Since it is a self-assessment by volunteer institution adoption, the information is insufficient regarding efficiency, validity and reliability. |
Maturity
|
It does not describe the level of maturity. |
Source: Motiejunaite et al. (2014Motiejunaite, A., Delhaxhe, A., Balcon, M.-P., & Borodankova, O. (2014). La educación y atención a la primera infancia. Agencia Ejecutiva en el Ámbito Educativo, Audiovisual y Cultural.) and Pimenta (2017Pimenta, C. O. (2017). Avaliações municipais da educação infantil: Contribuições para a garantia do direito à educação das crianças brasileiras? [Tese de doutorado, Universidade de São Paulo - Faculdade de Educação].). |
QUALITÀ LÚDICA - Italy |
Type
|
Self-evaluation |
Methodology
|
Systematic observation of the spaces and data collection by evaluators on the expansion of the repertoire of games and the playful behaviors and competencies of the children. |
Indicators
|
Educational project; space (in the room, in the internal and external common areas); materials (in the room, in the internal and external common areas); time; group composition; adult and game. |
Time
|
Application time is not described. |
Participation
|
Promotes participation from the perspective of context assessment. |
Results
|
The data collected are discussed with the external evaluator and the segments of the educational community to reach a consensus. |
Post-results
|
Not described. |
Viability
|
Not described, requires knowledge about child development and game-based activities, requiring greater qualification and mediation of the external evaluator. |
Validity and reliability
|
Subjective evaluation depends on the interpretation of the evaluators. The instrument does not present sufficient information on the degree of reliability. |
Maturity
|
Level of maturity is not described. Still a pilot/research project. |
Source: Moro and Souza (2016Moro, C., & Souza, G. de. (2016). Para uma análise pedagógica dos contextos educativos. (Entrevista com Anna Bondioli, Monica Ferrari, Donatella Savio). Universidade de Pávia/Itália.). |
ERVIS |
Type
|
Self-evaluation - inclusive education |
Methodology
|
Comprised of five areas evaluated, subdivided into 41 items, observation of spaces performed by professionals preferably trained in inclusive education, but improvements target all children. |
Indicators
|
Structure; qualification; professional activities; relationship among adults; integration process. |
Time
|
Application time is not described. |
Participation
|
Participatory, dialog- based with the professionals of the institution. |
Results
|
Data targets identifying the presence or absence of significant factors for the education of children with disabilities. |
Post-results
|
With the results, a specific project is drafted, with participation by faculty in proposing, implementing, evaluating, and modifying personalized paths designed to teach children with “disabilities”. |
Viability
|
Contextual, requires specific training in special education. |
Validity and reliability
|
Insufficient information is provided on the degree of reliability, since it depends on the interpretation of the evaluators, as well as the mastery of inclusive processes. |
Maturity
|
The level of maturity could not be identified. |
Source: Bondioli (2009Bondioli, A. (2009). ERVIS: Elementi per Rilevare e Valutare l’integrazione Scolastica. Editora Junior.) and Pimenta (2017Pimenta, C. O. (2017). Avaliações municipais da educação infantil: Contribuições para a garantia do direito à educação das crianças brasileiras? [Tese de doutorado, Universidade de São Paulo - Faculdade de Educação].). |
SPRING - Italy |
Type |
Self-evaluation and hetero-evaluation |
Methodology
|
External evaluator makes the initial contact, meetings, interviews and observations of spaces and practices, preparation of the final report on the data collected, debated, and analyzed for the return to the institution. With dimensions that relate to each other, they serve both for self-assessment and hetero-assessment. Quantifiable data is not used, being totally descriptive, requires time for subjectivity, including to consult the results. |
Indicators
|
Organization of the educational context; functioning of the working group; the institution’s relations with families and the community; evaluation processes. |
Time
|
On average six months. Reapplication depends on the progress achieved in the improvement plan. |
Participation
|
Provides for the participation of the different segments of the institution, formative, dialog-based, debate and confrontation, including pedagogical concepts for the quality negotiated. |
Results
|
Drafting of improvement plan. |
Post-results
|
To put improvement plan into practice with the changes and continuities indicated. |
Viability
|
Viable, in a systemic perspective, with qualified human structure, of specific knowledge about the instrument and pedagogical conception to support the institution. |
Validity and reliability
|
Offers internal consistency, by presenting analysis of the context from the standpoint of different segments, providing validity and reliability. |
Maturity
|
Not described. Recently created instrument. |
Source: Marcuccio and Zanelli (2013Marcuccio, M., & Zanelli, P. (2013). Sguardi sul nido... Strumento per lo Sviluppo di Processi Riflessivi e Indagini valutative nei Nidi da parte dei Gruppi di lavoro educativi (SPRING). Edizioni Junior-Spaggiari Edizioni.) and Moro (2018Moro, C. (2018). Diferentes olhares para a creche: A avaliação de contexto com o instrumento SPRING em um município da Emilia Romagna. Revista Linhas, 19(40), 138-160.). |
EARLY CHILDHOOD EDUCATION QUALITY INDICATORS - Brazil |
Type
|
Self-evaluation |
Methodology
|
Participants are divided into seven teams, one for each dimension, with coordinator and reporter. The quality level is based on the colors of the semaphore. At the conclusion of this stage, the subgroups return to plenary and each dimension is summarized, in order to negotiate and reach a consensus on the outcome and prepare an improvement plan. |
Indicators
|
Institutional planning; multiplicity of experiences and languages; interactions; health promotion; spaces, materials, and furniture; faculty and other professionals qualifications and working conditions; cooperation and exchange with families and participation in social protection networks. |
Time
|
One day. |
Participation
|
Participation and positioning of the different segments of the institution is essential. |
Results
|
To identify the institution’s strengths and weaknesses for the purpose of drafting an action plan consisting of: dimension, indicator, problems, actions, responsible and deadlines. |
Post-results
|
Monitoring of the action plan by all segments is not proposed, nor does it propose attributions to the Secretaria Municipal de Educação [Municipal Department of Education] and/or public policies. |
Viability
|
Accessible and easy to apply. |
Validity and reliability
|
Not being formative interferes with validity and reliability. Depends on the design and recognition of the educational community to identify the quality of services. |
Maturity
|
Maturity level is not described but can be applied at different times and contexts. |
Source: MEC (2009). |