Acessibilidade / Reportar erro

Cultural adaptation of the MISSCARE tool for use in the Chilean context

Abstract

Objective

to validate the MISSCARE instrument for use in the Chilean context.

Method

sociolinguistic adaptation of MISSCARE involved translation and back-translation, followed by evaluation by English-speaking nurses. Finally, content validation was achieved through expert review, focusing on cultural and content adaptation.

Results

the 2.0 version of MISSCARE demonstrated both face and content validity. Missed care is an adverse event that threatens the safety and quality of health care. Having a validated instrument to measure missed care will serve as an indicator to improve the quality of health care provided.

Conclusions and implications for practice

the adapted version of MISSCARE is a reliable, valid tool for measuring missed nursing care in hospitals. The study of missed nursing care will generate essential knowledge to improve the quality of health care and to develop management indicators. These insights will drive strategies and interventions in order to optimize services and ensure a higher standard of care.

Keywords:
Nursing Care; Nursing Research; Patient Safety; Quality of Health Care; Validation Study

Resumen

Objetivo

validar el instrumento de cuidado perdido de enfermería (MISSCARE), para ser utilizado en contexto chileno.

Método

adaptación sociolingüística de MISSCARE a través de traducción y retrotraducción, seguida por evaluación de profesionales de enfermería con manejo de idioma inglés y finalmente validación de contenido a través de revisión, adecuación cultural y de contenido, por expertos del área.

Resultados

el instrumento en su versión 2.0 demostró validez aparente y de contenido. El cuidado de enfermería perdido es un evento adverso que amenaza la seguridad y calidad de la atención en salud. Contar con un instrumento validado que mida el cuidado perdido será un indicador para mejorar la calidad en la atención prestada.

Conclusiones e implicaciones para la práctica

el instrumento MISSCARE en su versión adaptada es una herramienta confiable y válida para medir el cuidado perdido en instituciones hospitalarias. Estudiar los cuidados omitidos generará conocimientos clave para mejorar la calidad de la atención en salud y desarrollar indicadores de gestión. Estos conocimientos impulsarán estrategias e intervenciones para optimizar servicios y garantizar un estándar más alto en la atención.

Palabras clave:
Cuidado de Enfermería; Calidad de la Atención de Salud; Estudio de Validación; Investigación en Enfermería; Seguridad del Paciente

Resumo

Objetivo

validar o instrumento MISSCARE para uso no contexto chileno.

Método

a adaptação sociolinguística do MISSCARE envolveu tradução e retrotradução, seguida por avaliação de enfermeiros falantes de inglês. Finalmente, a validação de conteúdo foi alcançada por meio de revisão por especialistas, com foco na adaptação cultural e de conteúdo.

Resultados

a versão 2.0 do MISSCARE demonstrou validade de face e de conteúdo. O cuidado perdido é um evento adverso que ameaça a segurança e a qualidade da assistência à saúde. Ter um instrumento validado para medir o cuidado perdido servirá como um indicador para melhorar a qualidade da assistência à saúde prestada.

Conclusão e implicações para a prática

a versão adaptada do MISSCARE é uma ferramenta confiável e válida para medir o cuidado de enfermagem perdido em hospitais. O estudo do cuidado de enfermagem perdido gerará conhecimento essencial para melhorar a qualidade da assistência à saúde e desenvolver indicadores de gestão. Esses insights impulsionarão estratégias e intervenções para otimizar os serviços e garantir um padrão mais elevado de atendimento.

Palavras-chave:
Cuidados de Enfermagem; Estudo de Validação; Pesquisa em Enfermagem; Qualidade da Assistência à Saúde; Segurança do Paciente

INTRODUCTION

Patient safety has become a major focus in recent decades. Various studies show health care systems are prone to errors, resulting in a high risk of adverse events, which occur in 3.7%–16.6% of all hospital admissions.11 Leapel LL, Brennan TA, Laird N, Hebert L, Localio AR, Lawthers AG. The nature of adverse events in hospitalized patients: results of the Harvard Medical Practice Study II. Am J Infect Control. 1991;20(6):332. http://doi.org/10.1016/S0196-6553(05)80240-8.
http://doi.org/10.1016/S0196-6553(05)802...

That is why the World Health Organization (WHO) launched the World Alliance for Patient Safety in 200422 World Health Organization, World Alliance for Patient Safety, Research Priority Setting Working Group. Summary of the evidence on patient safety: implications for research [Internet]. Geneva: WHO; 2008 [citado 2023 mar 30]. Disponible en: https: //apps.who.int/iris/handle/10665/43874
https: //apps.who.int/iris/handle/10665/...
and the European Commission adopted the Luxembourg Declaration on Patient Safety in 2005, which recognizes access to quality health care as a fundamental human right.33 European Commission, DG Health and Consumer Protection. Declaración de Luxemburgo en seguridad del paciente [Internet]. Luxemburgo; 2005 [citado 2023 mar 30]. Disponible en: https://eur-lex.europa.eu/legal-content/ES/TXT/PDF/?uri=CELEX:52008PC0837R(01)&qid=1720397448945
https://eur-lex.europa.eu/legal-content/...

This alarming situation has led various international and national organizations to formulate strategies to maintain quality standards within healthcare facilities. Among these strategies, the Chilean Ministry of Health, based on the 2004 health reform, has promoted an accreditation system for institutional providers with the aim of achieving continuous improvement in the quality and safety of health care processes.44 Febré N, Mondaca-Gómez K, Méndez-Celis P, Badilla-Morales V, Soto-Parada P, Ivanovic P. Calidad en enfermería: su gestión, implementación y medición. Rev Med Clin Las Condes. 2018;29(3):278-87. http://doi.org/10.1016/j.rmclc.2018.04.008.
http://doi.org/10.1016/j.rmclc.2018.04.0...
This leads to the concept of patient safety, which is defined as the “reduction of risk of unnecessary harm associated with health care to an acceptable minimum” with the aim of avoiding adverse events that endanger the lives of health care users.55 World Health Organization. Más que palabras: marco conceptual de la Clasificación Internacional para la Seguridad del Paciente [Internet]. Geneva: WHO; 2009. Informe Técnico Definitivo [citado 2019 mar 30]. Disponible en: http://www.calidad.salud.gob.mx/site/calidad/docs/dsp-sp_00E.pdf
http://www.calidad.salud.gob.mx/site/cal...

To meet the Chilean National Accreditation System for Institutional Health Providers standards, Chilean health care facilities must have a system for monitoring adverse events related to health care. This system should include a review of practices in response to such events and the implementation of preventive measures that have been shown to be effective.44 Febré N, Mondaca-Gómez K, Méndez-Celis P, Badilla-Morales V, Soto-Parada P, Ivanovic P. Calidad en enfermería: su gestión, implementación y medición. Rev Med Clin Las Condes. 2018;29(3):278-87. http://doi.org/10.1016/j.rmclc.2018.04.008.
http://doi.org/10.1016/j.rmclc.2018.04.0...

One of the key success factors in improving patient safety currently is effective teamwork among health care workers and good communication between them and their patients. In terms of human factor, nurses are globally recognized as the largest workforce in health care.66 Chile, Superintendencia de Salud. Sistemas de vigilancia y notificación de Eventos Adversos [Internet]. Santiado: Unidad Técnica Asesora, Superintendencia de Salud; 2011 [citado 2019 mar 30]. Disponible en: http://www.supersalud.gob.cl/observatorio/671/articles-6921_recurso_1.pdf
http://www.supersalud.gob.cl/observatori...

As a result, the nursing team is the most involved in the health care process, making them critical to the safety and quality of patient care due to their proximity to the patient. They are also key players in the prevention of adverse events.77 World Health Organization. Health workforce: global strategic directions for nursing and midwifery report by the Director-General [Internet]. Geneva: WHO; 2020 [citado 2023 mar 30]. Disponible en: https://apps.who.int/gb/ebwha/pdf_files/WHA74/A74_13-en.pdf
https://apps.who.int/gb/ebwha/pdf_files/...
Nurses are accountable for the quality of care they provide, so it is imperative to identify any gaps in care and take appropriate action, including restructuring the services provided, to address the issue of missed care.88 Wakefield BJ. Enfrentando la realidad del cuidado perdido. BMJ Qual Saf. 2014;23:92-4. http://doi.org/10.1136/bmjqs-2013-002489.
http://doi.org/10.1136/bmjqs-2013-002489...

Patient safety in health care is a concept that is seen as unifying towards a culture of quality, as it leads to the development of the nursing discipline. Patient safety requires professionals to systematically evaluate care, thereby positioning the role of the nurse as an important part of the health care team.99 Kalisch BJ. Missed nursing care: a qualitative study. J Nurs Care Qual. 2006;21(4):306-13. http://doi.org/10.1097/00001786-200610000-00006. PMid:16985399.
http://doi.org/10.1097/00001786-20061000...

Few studies have addressed a critical element of quality of care that is intimately related to the safety and quality of the care provided: the omission or non-provision of care.99 Kalisch BJ. Missed nursing care: a qualitative study. J Nurs Care Qual. 2006;21(4):306-13. http://doi.org/10.1097/00001786-200610000-00006. PMid:16985399.
http://doi.org/10.1097/00001786-20061000...
,1010 World Health Organization. Patient safety [Internet]. Geneva: WHO; 2020 [citado 2020 abr 29]. Disponible en: https://www.who.int/patientsafety/en/
https://www.who.int/patientsafety/en/...

Incidents that occur while providing care, such as giving a patient the wrong dose of medication, are called errors of commission. Procedures that are not performed according to the patient’s needs, such as assisting with feeding, are called errors of omission (actions that are either partially or completely omitted or delayed).99 Kalisch BJ. Missed nursing care: a qualitative study. J Nurs Care Qual. 2006;21(4):306-13. http://doi.org/10.1097/00001786-200610000-00006. PMid:16985399.
http://doi.org/10.1097/00001786-20061000...

This definition led to the first articulation of the concept of missed care by Kalisch in 2006. Through a qualitative study involving 173 nursing professionals, she sought to identify lapses in various nursing interventions in the hospital setting. This was the beginning of this theoretical framework, as the study found that many interventions were often missed due to reasons such as staff shortages, lack of time, poor teamwork, etc.99 Kalisch BJ. Missed nursing care: a qualitative study. J Nurs Care Qual. 2006;21(4):306-13. http://doi.org/10.1097/00001786-200610000-00006. PMid:16985399.
http://doi.org/10.1097/00001786-20061000...

Kalisch validated MISSCARE in 2009 to assess the impact of missed nursing care. Her validation study found 70% of missed care involved nursing interventions and planning for those interventions.1111 Kalisch BJ, Landstrom G, Williams RA. Missed nursing care: errors of omission. Nurs Outlook. 2009;57(1):3-9. http://doi.org/10.1016/j.outlook.2008.05.007. PMid:19150261.
http://doi.org/10.1016/j.outlook.2008.05...

In order to address the problems arising from the provision of care, and in line with current global trends and globalization, it is necessary to develop tools in the health care sector that systematically evaluate the care provided. The goal is to ensure and optimize services while reducing the risk of failures. One of these tools is the clinical nursing record, a fundamental instrument that should be oriented towards the permanent documentation of care. These records, based on an ethical-legal framework, should be considered as an indicator of the quality of care.1212 Monteiro C, Machado AF, Gonçalves PM. Interruptions of nurses’ activities and patient safety: an integrative literature review. Rev Lat Am Enfermagem. 2015;23(1):169. http://doi.org/10.1590/0104-1169.0251.2539.
http://doi.org/10.1590/0104-1169.0251.25...

From the analysis of the reported background, and in line with the Chile’s health strategies to improve health care safety, the need for a tool to quantify the extent of nursing errors of omission arose. As nurses are the largest workforce in health care facilities, their errors of omission can have a significant impact on patient safety and health care costs. In addition, although MISSCARE does exist, it has not been validated for the Chilean context. Therefore, it is essential to have a validated instrument that examines health care workers’ perceptions of lapses in health care and the reasons for such lapses, as these directly influence the occurrence of adverse events, thereby jeopardizing patient safety.

From these considerations, the objective of the present study is to determine if MISSCARE is a valid instrument for the assessment of missed nursing care in the Chilean context.

LITERATURE REVIEW

Missed care, defined as any aspect of care that is either wholly or partially omitted or significantly delayed, poses a health risk.1313 Kalisch B, Landstrom G, Hinshaw A. Missed nursing care: a concept analysis. J Adv Nurs. 2009;65(7):1509-17. http://doi.org/10.1111/j.1365-2648.2009.05027.x. PMid:19456994.
http://doi.org/10.1111/j.1365-2648.2009....
In the United States,99 Kalisch BJ. Missed nursing care: a qualitative study. J Nurs Care Qual. 2006;21(4):306-13. http://doi.org/10.1097/00001786-200610000-00006. PMid:16985399.
http://doi.org/10.1097/00001786-20061000...
a qualitative study identified some of the care activities that are regularly omitted, and the reasons given for not providing such care. This study served as the basis for the development and validation of MISSCARE.

Having a validated tool such as MISSCARE is essential because it allows for the quantification of missed nursing care and the reasons for those omissions.

The omission of care can be due to a variety of reasons, including the nursing staff’s failure to prioritize this care or the perception the patient can do it independently or with the help of family members.99 Kalisch BJ. Missed nursing care: a qualitative study. J Nurs Care Qual. 2006;21(4):306-13. http://doi.org/10.1097/00001786-200610000-00006. PMid:16985399.
http://doi.org/10.1097/00001786-20061000...

In Chile, there are no studies related to the lack of care. Literature search revealed instruments to measure 1) caring behaviors (Caring Behaviors Assessment [CBA] tool),1414 Ayala RA, Calvo MJ. Cultural adaptation and validation of the Caring Behaviors Assessment tool in Chile. Nurs Health Sci. 2017;19(4):459-66. http://doi.org/10.1111/nhs.12364.
http://doi.org/10.1111/nhs.12364...
2) humanized care (Perception of Humanized Nursing Care [PHCE] tool),1515 Hermosilla Ávila A, Mendoza Llanos R, Contreras Contreras S. Instrumento para valoración del cuidado humanizado brindado por profesionales de enfermería a personas hospitalizadas. Index Enferm [Internet]. 2016 [citado 2020 abr 29];25(4):273-7. Disponible en: http://scielo.isciii.es/scielo.php?script=sci_arttext&pid=S1132-12962016000300011&lng=es&tlng=es
http://scielo.isciii.es/scielo.php?scrip...
3) perception of care quality (Karen Personal tool),1616 Luengo-Martínez C, Paravic-Klijn T, Burgos-Moreno M, López-Espinoza MÁ. Adaptación transcultural del instrumento Karen Personal para medir la percepción de la calidad del cuidado en enfermeras de hospital. Enferm Clin. 2019;29(3):146-54. http://doi.org/10.1016/j.enfcli.2018.10.007.
http://doi.org/10.1016/j.enfcli.2018.10....
4) nursing workload in intensive care units (Nursing Activities Score [NAS] tool),1717 Toffoletto MC, Reynaldos Grandón KL, Molina Muñoz Y, Grillo Padilha K. Validación del Nursing activities score en unidades de cuidados intensivos chilenas. Cienc Enferm. 2018;24:13-5. http://doi.org/10.4067/S0717-95532018000100213.
http://doi.org/10.4067/S0717-95532018000...
5) transpersonal human care,1818 Poblete-Troncoso M, Valenzuela-Suazo S, Merino J-M. Validación de dos escalas utilizadas en la medición del cuidado humano transpersonal basadas en la Teoría de Jean Watson. Aquichan. 2012;12(1):8-21. http://doi.org/10.5294/aqui.2012.12.1.1.
http://doi.org/10.5294/aqui.2012.12.1.1...
etc. However, none of these instruments address the construct of missed care in the way MISSCARE does.

Besides, the scales identified have limitations in that they only measure workload or the perception of care by both health care workers and users. They do not include whether care is omitted or determine the reasons for such omissions. These reasons justify the need to develop a new instrument adapted to the phenomenon under study, thus addressing one of the indicators that allows the evaluation of the quality and safety of care processes, which is the lack of care.

During this literature review, using keywords such as quality, safety of care, job satisfaction, and work environment, and after discarding articles based on year of publication and those that did not address the topic of missed care, references were found related to the MISSCARE tool developed by Kalisch and the analysis of this phenomenon.1919 Krupowicz J, Kuropka I. Convergence of population structures of the european union member states: the past and the future. Sustainability. 2022;14(2):1024. http://doi.org/10.3390/su14021024.
http://doi.org/10.3390/su14021024...

20 Chaboyer W, Harbeck E, Lee BO, Grealish L. Missed nursing care: an overview of reviews. Kaohsiung J Med Sci. 2021;37(2):82-91. http://doi.org/10.1002/kjm2.12308.
http://doi.org/10.1002/kjm2.12308...

21 Siqueira LDC, Caliri MHL, Kalisch B, Dantas RAS. Cultural adaptation and internal consistency analysis of the MISSCARE Survey for use in Brazil. Rev Lat Am Enfermagem. 2013;21(2):610-7. http://doi.org/10.1590/S0104-11692013000200019. PMid:23797556.
http://doi.org/10.1590/S0104-11692013000...

22 Nymark C, Göransson KE, Saboonchi F, Falk AC, Von Vogelsang AC. Translation, culture adaption and psychometric testing of the MISSCARE survey-Swedish version. J Clin Nurs. 2020;29(23-24):4645-52. http://doi.org/10.1111/jocn.15505. PMid:32956504.
http://doi.org/10.1111/jocn.15505...
-2323 Lisby M, Tei R, Mainz H, Gregersen M, Andersen KV. A Danish version of the MISSCARE survey: translation and validation. Scand J Caring Sci. 2023;37(1):301-12. http://doi.org/10.1111/scs.13143. PMid:36598034.
http://doi.org/10.1111/scs.13143...

Since there are no Chilean studies on missed nursing care, and it is a global phenomenon that poses a risk to patient safety, studies that validate this construct and provide a valid instrument adapted to the local context are essential.

Besides measuring the extent of these failures, MISSCARE will provide valuable information for developing strategies to mitigate missed nursing care. MISSCARE will help to determine the actual impact that shifts, and their organization have on the organizational climate and the resulting impact on quality of care and patient safety. These findings will also allow the investigation of possible links with the nursing staff shortage, which still needs to be estimated in Chile.

METHOD

This methodological study was approved by the Ethics Committee of the Vice Rectorate for Research and Development at the University of Concepción (code CEBB 724-2020) and the Scientific Ethics Committee of the SSC (code SSC:20-12-63). Permission for cultural adaptation was obtained from Beatrice Kalisch, the original author of MISSCARE.

MISSCARE is a quantitative, self-administered instrument for nurses and nursing aides which takes 10 to 15 minutes to fill out. MISSCARE consists of 64 items divided into three sections. First section includes sociodemographic and work-related data from nurses, with a total of 23 questions. Subsequent sections consist of different items, with 24 elements of basic nursing care for hospitalized patients in Part A and 17 reasons for missed nursing care in Part B.1111 Kalisch BJ, Landstrom G, Williams RA. Missed nursing care: errors of omission. Nurs Outlook. 2009;57(1):3-9. http://doi.org/10.1016/j.outlook.2008.05.007. PMid:19150261.
http://doi.org/10.1016/j.outlook.2008.05...

Second section, called “Section A: Missed Nursing Care,” refers to all elements of care provided to the patient. This section is divided into four dimensions: assessment, interventions — individual needs, interventions — basic care, and planning. Section A uses a 5-point Likert scale with response options from higher to lower: 5 for “always,” 4 for “frequently,” 3 for “occasionally,” 2 for “rarely,” 1 for “never,” and 0 for “non applicable.” The non-applicable option is included for nursing activities that are not completed on the night shift, such as ambulating with and feeding patients.1111 Kalisch BJ, Landstrom G, Williams RA. Missed nursing care: errors of omission. Nurs Outlook. 2009;57(1):3-9. http://doi.org/10.1016/j.outlook.2008.05.007. PMid:19150261.
http://doi.org/10.1016/j.outlook.2008.05...

The third section, called “Section B: Reasons for Missed Care,” refers to all the factors that lead to missed care, such as staff shortage, patient emergencies, and an increase in the number of patients, etc. This section is divided into three factor dimensions: communication, material resources, and labor resources. The response range is a 4-point Likert scale with responses from highest to lowest: 4 for “significant factor,” 3 for “moderate factor,” 2 for “minor factor,” and 1 for “not a reason for unmet nursing care.” The minimum score for this section is 17 and the maximum is 68, with higher scores indicating significant reasons for lack of care.1111 Kalisch BJ, Landstrom G, Williams RA. Missed nursing care: errors of omission. Nurs Outlook. 2009;57(1):3-9. http://doi.org/10.1016/j.outlook.2008.05.007. PMid:19150261.
http://doi.org/10.1016/j.outlook.2008.05...

To validate the original instrument and develop a valid, reliable tool to measure missed care and the reasons for it, the authors of the original version conducted tests of construct validity, internal consistency, and stability (test-retest) before finalizing the version. They distinguished five phases in the development of the instrument: qualitative study, analysis of the concept of missed care, instrument development, data collection, and psychometric testing of MISSCARE.1111 Kalisch BJ, Landstrom G, Williams RA. Missed nursing care: errors of omission. Nurs Outlook. 2009;57(1):3-9. http://doi.org/10.1016/j.outlook.2008.05.007. PMid:19150261.
http://doi.org/10.1016/j.outlook.2008.05...

Procedure for cultural adaptation

For the cultural adaptation of MISSCARE, following the recommendations of the WHO2424 Wild D, Grove A, Martin M, Eremenco S, McElroy S, Verjee-Lorenz A et al. Principles of good practice for the translation and cultural adaptation process for Patient-Reported Outcomes (PRO) measures: report of the ISPOR Task Force for Translation and Cultural Adaptation. Value Health. 2005;8(2):94-104. http://doi.org/10.1111/j.1524-4733.2005.04054.x. PMid:15804318.
http://doi.org/10.1111/j.1524-4733.2005....
and research on guidelines for the cross-cultural adaptation process,2525 Carvajal A, Centeno C, Watson R, Martínez M, Sanz Rubiales Á. ¿Cómo validar un instrumento de medida de la salud? An Sist Sanit Navar. 2011;34(1):63-72. http://doi.org/10.4321/S1137-66272011000100007. PMid:21532647.
http://doi.org/10.4321/S1137-66272011000...
,2626 Rivero-Núñez E, González-Cárdenas L, Bayarre-Vea H. Validación y adaptación lingüística-cultural del Instrumento World Health Organization Quality of Live Bref (WHOQOL) en mujeres de edad mediana. Rev Cubana Enferm [Internet]. 2016 [citado 2020 abr 29];32(4):126-39. Disponible en: http://scielo.sld.cu/scielo.php?script=sci_arttext&pid=S0864-03192016000400017&lng=es&tlng=es
http://scielo.sld.cu/scielo.php?script=s...
the proposed high-quality methodology was established. This methodology includes the translation and back-translation of the instrument by bilingual nursing professionals who first evaluated whether missed nursing care does exist in the Chilean culture.

Direct translation is performed from the original language to the target language. This step was performed independently by two bilingual nurses with academic and/or clinical experience. They were unaware of the specific topic but knew the aim of the study. They then compared the two translations and agreed on the first Spanish version of the instrument. The back-translation was done by a bilingual nurse who was familiar with the topic but not with the scale. This version was then compared with the original English version to make final adjustments.2424 Wild D, Grove A, Martin M, Eremenco S, McElroy S, Verjee-Lorenz A et al. Principles of good practice for the translation and cultural adaptation process for Patient-Reported Outcomes (PRO) measures: report of the ISPOR Task Force for Translation and Cultural Adaptation. Value Health. 2005;8(2):94-104. http://doi.org/10.1111/j.1524-4733.2005.04054.x. PMid:15804318.
http://doi.org/10.1111/j.1524-4733.2005....

For content validity, the translated version was tested with a sample of 10 nurse experts from academia and/or clinical practice, following literature recommendations for the appropriate use of the Delphi method. After completing the instrument, the researcher interviewed them to explore their understanding of each item. Besides, they were asked to provide a general quantitative evaluation of the instrument as well as positive aspects, limitations, and any recommendations they felt were necessary.

The aspects evaluated for each item were clarity, sufficiency, coherence, and relevance, each rated on a scale of 1 to 4 (in which 1: does not meet the criterion and 4: high level of compliance). Volunteers’ responses were analyzed, and the draft was further adjusted to ensure readability through simple language.

In the expert review of the instrument and proposed changes for the version adapted to the Chilean population, the criterion was that each item should have an agreement rating greater than 3. Expected values were a mean > 2.5 and high scores (3-4) > 80% and/or a standard deviation < 0.90. All of these tests were part of the cultural adaptation of instruments originally written in English.

After being informed by the research team and providing informed consent, each participant was assigned a code to ensure anonymity. The expert group participants were asked to provide suggestions through an online Google form within 15 days of reading all of the documentation provided. In this form, they had to indicate the concordance between each item of the translated instrument (MC_ESP_1/2) and the back-translated Chilean version (MC_ESP_ENG_1/2).

With the obtained instrument and after the modifications, a preliminary version adapted to the Chilean context is now available. This version will be applied to a selected population in the next phase of this research, which includes the validation of the instrument, to establish the relevance, validity, and suitability of validated MISSCARE in the Chilean context.

The next step follows the classic steps of instrument validation, following the previously analyzed face and content validity. It involves conducting a study with a validation sample to determine the psychometric properties of the instrument. The process begins by describing the sociodemographic characteristics of the sample, as these characteristics define the population on which the instrument validation was conducted.

RESULTS

Ten experts participated in the pilot test and met the selection criteria: nurses with more than 10 years of professional experience in academia and clinical practice (hospital or community) and proficiency in English. Profile of experts is summarized in Table 1.

Table 1
Profiles of the expert group members.

First phase: obtaining the Chilean version of MISSCARE (MC_V1)

Analysis of the first group of experts to evaluate the practical applicability of first version of the Chilean version of MISSCARE concluded with a 95% consensus on the equivalence of the Chilean version of MISSCARE with the original instrument for 90% of the items. In this phase, irrelevant items were removed, missing items were added, and some items were adapted to Spanish (Chilean) wording. There was no need to consult the author of the instrument to clarify any uncertainties.

The Tentative Instrument “MC_V1,” which underwent the following modifications, became the Chilean version of MISSCARE:

  • Added item: Name of the hospital.

  • Added item: Type of hospital, as there are public and private hospitals in Chile.

  • Added item: Type of unit, considering the complexity of the patient.

  • Modified and adapted items related to level of education to the Chilean context.

  • The age scale was modified to indicate full years.

  • Adapted item: Shifts and working hours to the Chilean context.

  • Added item: Employment status.

  • Added item: Personal workload.

  • Changed wording of item: Hand hygiene to indicate that it refers to the responsible nurse.

  • In summary, MC_V1 increased 6 items in the first part, from 20 to 26, and retained the same number in sections A (24) and B (17).

  • The tentative MC_V1 questionnaire is shown in Supplement 1 Supplement 1 - Missed Nursing Care MC_V1. .

The process of obtaining MC_V1 is shown in Figure 1 below.

Figure 1
Schematic of the translation and back-translation process of MISSCARE.

Second phase: obtaining the Chilean version of MISSCARE (MC_V2)

After receiving MC_V1, four nurses from the first phase were convened to form a second expert group. The aim was to correct possible discrepancies due to the translation and cultural adaptation process, thus assessing the face (logical) and content validity of MC_V1. The profile of the second group of experts is described in Table 2.

Table 2
Profiles of the expert group members.

For face and content validity, the second group of experts assessed whether the items of the MC_V1 instrument covered all aspects of the perception of missed care (content validity) and whether the items were clear and unambiguous (face validity). This phase is shown in Figure 2.

Figure 2
Schematic of the translation and cultural adaptation process of MISSCARE (2nd part).

In light of these observations, the second group of experts suggested a number of changes to improve the face and content validity of MC_V1.

The proposed modifications are detailed as follows:

  • Add item: Nursing specialty.

  • Modify the wording of some items to adapt to the terminology used in Chile: hygiene/bath, elimination needs.

  • Split item 9 into two (differentiating patient education and emotional support).

  • Add a new item for emotional support and/or education for family members.

  • Split item 19 into two (differentiating care of peripheral lines and central lines).

  • Add item (27) on the perception of missed respiratory care.

  • In Section B of the questionnaire, add item (18) on the level of importance of excessive administrative workload in relation to the omission of care.

  • Recommended improving the instructions.

After this process, a second version of the tentative questionnaire, called MC_V2, was obtained and approved by 100% of the experts who participated in this phase. This second version consists of a total of 27 statements in Section A (the original English version has 24) and 18 statements in Section B (compared to 17 in the original instrument). This version is presented in Supplement 2 Supplement 2 - Missed Nursing Care MC_V2. .

At this stage, the MC_V2 instrument, the final tentative version, is obtained. It will be used in the second stage for the validation study in a selected population.

DISCUSSION

The process of cultural adaptation of the Chilean version of MISSCARE was performed according to the recommendations in the literature,2424 Wild D, Grove A, Martin M, Eremenco S, McElroy S, Verjee-Lorenz A et al. Principles of good practice for the translation and cultural adaptation process for Patient-Reported Outcomes (PRO) measures: report of the ISPOR Task Force for Translation and Cultural Adaptation. Value Health. 2005;8(2):94-104. http://doi.org/10.1111/j.1524-4733.2005.04054.x. PMid:15804318.
http://doi.org/10.1111/j.1524-4733.2005....

25 Carvajal A, Centeno C, Watson R, Martínez M, Sanz Rubiales Á. ¿Cómo validar un instrumento de medida de la salud? An Sist Sanit Navar. 2011;34(1):63-72. http://doi.org/10.4321/S1137-66272011000100007. PMid:21532647.
http://doi.org/10.4321/S1137-66272011000...
-2626 Rivero-Núñez E, González-Cárdenas L, Bayarre-Vea H. Validación y adaptación lingüística-cultural del Instrumento World Health Organization Quality of Live Bref (WHOQOL) en mujeres de edad mediana. Rev Cubana Enferm [Internet]. 2016 [citado 2020 abr 29];32(4):126-39. Disponible en: http://scielo.sld.cu/scielo.php?script=sci_arttext&pid=S0864-03192016000400017&lng=es&tlng=es
http://scielo.sld.cu/scielo.php?script=s...
thereby ensuring its correct use. This article presents the face and content validity of the adapted version. The stages of validation of the psychometric properties will be the subject of another publication.

For the process, the following phases were established: 1) direct translation; 2) back-translation; 3) cultural adaptation (first expert group); 4) face and content validity (second expert group).

The cultural adaptation process was conducted following the suggested steps of direct translation and back-translation to achieve the highest consensus.2424 Wild D, Grove A, Martin M, Eremenco S, McElroy S, Verjee-Lorenz A et al. Principles of good practice for the translation and cultural adaptation process for Patient-Reported Outcomes (PRO) measures: report of the ISPOR Task Force for Translation and Cultural Adaptation. Value Health. 2005;8(2):94-104. http://doi.org/10.1111/j.1524-4733.2005.04054.x. PMid:15804318.
http://doi.org/10.1111/j.1524-4733.2005....
,2626 Rivero-Núñez E, González-Cárdenas L, Bayarre-Vea H. Validación y adaptación lingüística-cultural del Instrumento World Health Organization Quality of Live Bref (WHOQOL) en mujeres de edad mediana. Rev Cubana Enferm [Internet]. 2016 [citado 2020 abr 29];32(4):126-39. Disponible en: http://scielo.sld.cu/scielo.php?script=sci_arttext&pid=S0864-03192016000400017&lng=es&tlng=es
http://scielo.sld.cu/scielo.php?script=s...
In order to perform this phase, comparisons were made with other validation studies of MISSCARE, which did not provide details on the expert group. Therefore, an average number of experts was determined based on what was presented in a Spanish study.

Thus, the expert group consisted of nursing professionals, all of whom had more than 10 years of professional experience, in addition to intermediate/advanced knowledge of English and experience in both clinical and teaching settings. There were 10 experts for the first phase and four for the second phase. This provided an adequate number of qualified experts to ensure the appropriateness of the instrument through evaluation and judgement. This approach achieved a consensus that allowed verification of the face and content validity of the instrument.

Recommendations from the experts included changing terminology, verbs, and statements, and adding new items to the instrument to facilitate understanding at the national level. Some of these recommendations included adding items such as type of hospital, type of unit, employment status, and personal workload. In addition, changes were made such as translating the acronym “PRN medication requests” to SOS for its Spanish equivalent, suggesting gender-neutral nomenclature, and translating academic titles to the national context. All of these changes improve the clarity of the tool. Similar research has shown the importance of modifying the terminology of some instruments to ensure a proper understanding of what is being asked and assessed.2727 Ramada-Rodilla JM, Serra-Pujadas C, Delclós-Clanchet GL. Adaptación cultural y validación de cuestionarios de salud: revisión y recomendaciones metodológicas. Salud Publica Mex. 2013;55(1):57-66. http://doi.org/10.1590/S0036-36342013000100009. PMid:23370259.
http://doi.org/10.1590/S0036-36342013000...
,2828 Ferreira M, Haas V, Dantas R, Felix M, Galvão C. Cultural adaptation and validation of an instrument on barriers for the use of research results. Rev Lat Am Enfermagem. 2017;25(0):28-52. http://doi.org/10.1590/1518-8345.1652.2852.
http://doi.org/10.1590/1518-8345.1652.28...

The correct translation and cultural adaptation of an instrument does not always guarantee the preservation of its psychometric properties. Validation in the target language and context is therefore essential. For an instrument to be considered valid, it must meet a number of criteria:2424 Wild D, Grove A, Martin M, Eremenco S, McElroy S, Verjee-Lorenz A et al. Principles of good practice for the translation and cultural adaptation process for Patient-Reported Outcomes (PRO) measures: report of the ISPOR Task Force for Translation and Cultural Adaptation. Value Health. 2005;8(2):94-104. http://doi.org/10.1111/j.1524-4733.2005.04054.x. PMid:15804318.
http://doi.org/10.1111/j.1524-4733.2005....
,2727 Ramada-Rodilla JM, Serra-Pujadas C, Delclós-Clanchet GL. Adaptación cultural y validación de cuestionarios de salud: revisión y recomendaciones metodológicas. Salud Publica Mex. 2013;55(1):57-66. http://doi.org/10.1590/S0036-36342013000100009. PMid:23370259.
http://doi.org/10.1590/S0036-36342013000...
,2828 Ferreira M, Haas V, Dantas R, Felix M, Galvão C. Cultural adaptation and validation of an instrument on barriers for the use of research results. Rev Lat Am Enfermagem. 2017;25(0):28-52. http://doi.org/10.1590/1518-8345.1652.2852.
http://doi.org/10.1590/1518-8345.1652.28...

  1. Reflect the theory in the phenomenon or concept being measured (construct validity).

  2. Accurately measure what it is intended to measure (criterion validity).

  3. Be consistent in its internal structure, with all of its items or groups of items pointing toward the same metric goal (internal consistency).

  4. Provide measurements that are free of error (reliability).

  5. Be capable of measuring change, both between individuals and in the same individual’s responses over time (sensitivity to change).

Having tools that cover different dimensions of care is essential for identifying problems associated with inadequate care. This approach allows for the establishment of a culture of quality aimed at improving the safety and quality of care provided. Therefore, this study is considered a valuable contribution to future research in this process. As shown, the validation of instruments that measure nursing care is a crucial aspect.2424 Wild D, Grove A, Martin M, Eremenco S, McElroy S, Verjee-Lorenz A et al. Principles of good practice for the translation and cultural adaptation process for Patient-Reported Outcomes (PRO) measures: report of the ISPOR Task Force for Translation and Cultural Adaptation. Value Health. 2005;8(2):94-104. http://doi.org/10.1111/j.1524-4733.2005.04054.x. PMid:15804318.
http://doi.org/10.1111/j.1524-4733.2005....
,2727 Ramada-Rodilla JM, Serra-Pujadas C, Delclós-Clanchet GL. Adaptación cultural y validación de cuestionarios de salud: revisión y recomendaciones metodológicas. Salud Publica Mex. 2013;55(1):57-66. http://doi.org/10.1590/S0036-36342013000100009. PMid:23370259.
http://doi.org/10.1590/S0036-36342013000...
,2828 Ferreira M, Haas V, Dantas R, Felix M, Galvão C. Cultural adaptation and validation of an instrument on barriers for the use of research results. Rev Lat Am Enfermagem. 2017;25(0):28-52. http://doi.org/10.1590/1518-8345.1652.2852.
http://doi.org/10.1590/1518-8345.1652.28...

Thus, after the next validation phase, this instrument, adapted to the Chilean population, will become a very useful tool in different settings, whether in public and/or private hospital contexts in Chile. This will make it possible to measure a highly relevant aspect today: missed nursing care, which is inextricably linked to quality of care and patient safety.99 Kalisch BJ. Missed nursing care: a qualitative study. J Nurs Care Qual. 2006;21(4):306-13. http://doi.org/10.1097/00001786-200610000-00006. PMid:16985399.
http://doi.org/10.1097/00001786-20061000...

CONCLUSIONS AND IMPLICATIONS FOR PRACTICE

Having validated tools is essential to establishing a culture of quality and safety in patient care. These tools not only improve quality, but also set standards of care that are in line with modern times. The instrument has demonstrated content and face validity according to experts, making it applicable to the Chilean context. The use of this instrument will provide updated information regarding the care of users. Although MISSCARE has demonstrated reliability and validity in different international contexts, its continuous adaptation and development is an evolving task, and future studies will be able to evaluate the applicability and strength of the instrument.

The background analyzed underscores the global concern about nursing errors and highlights the need for reliable and valid measures in this area. The significant contribution of nursing care to the overall quality of patient care underscores the importance of having adequate tools to measure errors of omission.

The adaptation and validation of an instrument, as described, involves several stages and requirements. Therefore, obtaining appropriate results will allow confidence in its application. Upon completion of the validation process, there will be an instrument applicable to the Chilean context that can serve as an indicator to evaluate care management. Missed nursing care has been shown to be an indicator of quality of care related to patient safety.

In addressing the limitations of the study, the validation was initially delayed due to the context of the COVID-19 pandemic. These circumstances led to delays and adjustments in the methodology, including changes in sample type and data collection. Another important point to note is that the population studied was limited to a single public hospital.

For future research on the evaluation of missed care, it is suggested that multicenter approaches be used. These studies could cover both public and private sectors and include professionals from different settings. This would allow for the assessment of potential differences in the context of health practice and provide a broader and more representative perspective of care provided in different settings.

Supplementary material

The following online material is available for this article:

Supplement 1 - Missed Nursing Care MC_V1.

Supplement 2 - Missed Nursing Care MC_V2.

Referencias

  • 1
    Leapel LL, Brennan TA, Laird N, Hebert L, Localio AR, Lawthers AG. The nature of adverse events in hospitalized patients: results of the Harvard Medical Practice Study II. Am J Infect Control. 1991;20(6):332. http://doi.org/10.1016/S0196-6553(05)80240-8
    » http://doi.org/10.1016/S0196-6553(05)80240-8
  • 2
    World Health Organization, World Alliance for Patient Safety, Research Priority Setting Working Group. Summary of the evidence on patient safety: implications for research [Internet]. Geneva: WHO; 2008 [citado 2023 mar 30]. Disponible en: https: //apps.who.int/iris/handle/10665/43874
    » https: //apps.who.int/iris/handle/10665/43874
  • 3
    European Commission, DG Health and Consumer Protection. Declaración de Luxemburgo en seguridad del paciente [Internet]. Luxemburgo; 2005 [citado 2023 mar 30]. Disponible en: https://eur-lex.europa.eu/legal-content/ES/TXT/PDF/?uri=CELEX:52008PC0837R(01)&qid=1720397448945
    » https://eur-lex.europa.eu/legal-content/ES/TXT/PDF/?uri=CELEX:52008PC0837R(01)&qid=1720397448945
  • 4
    Febré N, Mondaca-Gómez K, Méndez-Celis P, Badilla-Morales V, Soto-Parada P, Ivanovic P. Calidad en enfermería: su gestión, implementación y medición. Rev Med Clin Las Condes. 2018;29(3):278-87. http://doi.org/10.1016/j.rmclc.2018.04.008
    » http://doi.org/10.1016/j.rmclc.2018.04.008
  • 5
    World Health Organization. Más que palabras: marco conceptual de la Clasificación Internacional para la Seguridad del Paciente [Internet]. Geneva: WHO; 2009. Informe Técnico Definitivo [citado 2019 mar 30]. Disponible en: http://www.calidad.salud.gob.mx/site/calidad/docs/dsp-sp_00E.pdf
    » http://www.calidad.salud.gob.mx/site/calidad/docs/dsp-sp_00E.pdf
  • 6
    Chile, Superintendencia de Salud. Sistemas de vigilancia y notificación de Eventos Adversos [Internet]. Santiado: Unidad Técnica Asesora, Superintendencia de Salud; 2011 [citado 2019 mar 30]. Disponible en: http://www.supersalud.gob.cl/observatorio/671/articles-6921_recurso_1.pdf
    » http://www.supersalud.gob.cl/observatorio/671/articles-6921_recurso_1.pdf
  • 7
    World Health Organization. Health workforce: global strategic directions for nursing and midwifery report by the Director-General [Internet]. Geneva: WHO; 2020 [citado 2023 mar 30]. Disponible en: https://apps.who.int/gb/ebwha/pdf_files/WHA74/A74_13-en.pdf
    » https://apps.who.int/gb/ebwha/pdf_files/WHA74/A74_13-en.pdf
  • 8
    Wakefield BJ. Enfrentando la realidad del cuidado perdido. BMJ Qual Saf. 2014;23:92-4. http://doi.org/10.1136/bmjqs-2013-002489
    » http://doi.org/10.1136/bmjqs-2013-002489
  • 9
    Kalisch BJ. Missed nursing care: a qualitative study. J Nurs Care Qual. 2006;21(4):306-13. http://doi.org/10.1097/00001786-200610000-00006 PMid:16985399.
    » http://doi.org/10.1097/00001786-200610000-00006
  • 10
    World Health Organization. Patient safety [Internet]. Geneva: WHO; 2020 [citado 2020 abr 29]. Disponible en: https://www.who.int/patientsafety/en/
    » https://www.who.int/patientsafety/en/
  • 11
    Kalisch BJ, Landstrom G, Williams RA. Missed nursing care: errors of omission. Nurs Outlook. 2009;57(1):3-9. http://doi.org/10.1016/j.outlook.2008.05.007 PMid:19150261.
    » http://doi.org/10.1016/j.outlook.2008.05.007
  • 12
    Monteiro C, Machado AF, Gonçalves PM. Interruptions of nurses’ activities and patient safety: an integrative literature review. Rev Lat Am Enfermagem. 2015;23(1):169. http://doi.org/10.1590/0104-1169.0251.2539
    » http://doi.org/10.1590/0104-1169.0251.2539
  • 13
    Kalisch B, Landstrom G, Hinshaw A. Missed nursing care: a concept analysis. J Adv Nurs. 2009;65(7):1509-17. http://doi.org/10.1111/j.1365-2648.2009.05027.x PMid:19456994.
    » http://doi.org/10.1111/j.1365-2648.2009.05027.x
  • 14
    Ayala RA, Calvo MJ. Cultural adaptation and validation of the Caring Behaviors Assessment tool in Chile. Nurs Health Sci. 2017;19(4):459-66. http://doi.org/10.1111/nhs.12364
    » http://doi.org/10.1111/nhs.12364
  • 15
    Hermosilla Ávila A, Mendoza Llanos R, Contreras Contreras S. Instrumento para valoración del cuidado humanizado brindado por profesionales de enfermería a personas hospitalizadas. Index Enferm [Internet]. 2016 [citado 2020 abr 29];25(4):273-7. Disponible en: http://scielo.isciii.es/scielo.php?script=sci_arttext&pid=S1132-12962016000300011&lng=es&tlng=es
    » http://scielo.isciii.es/scielo.php?script=sci_arttext&pid=S1132-12962016000300011&lng=es&tlng=es
  • 16
    Luengo-Martínez C, Paravic-Klijn T, Burgos-Moreno M, López-Espinoza MÁ. Adaptación transcultural del instrumento Karen Personal para medir la percepción de la calidad del cuidado en enfermeras de hospital. Enferm Clin. 2019;29(3):146-54. http://doi.org/10.1016/j.enfcli.2018.10.007
    » http://doi.org/10.1016/j.enfcli.2018.10.007
  • 17
    Toffoletto MC, Reynaldos Grandón KL, Molina Muñoz Y, Grillo Padilha K. Validación del Nursing activities score en unidades de cuidados intensivos chilenas. Cienc Enferm. 2018;24:13-5. http://doi.org/10.4067/S0717-95532018000100213
    » http://doi.org/10.4067/S0717-95532018000100213
  • 18
    Poblete-Troncoso M, Valenzuela-Suazo S, Merino J-M. Validación de dos escalas utilizadas en la medición del cuidado humano transpersonal basadas en la Teoría de Jean Watson. Aquichan. 2012;12(1):8-21. http://doi.org/10.5294/aqui.2012.12.1.1
    » http://doi.org/10.5294/aqui.2012.12.1.1
  • 19
    Krupowicz J, Kuropka I. Convergence of population structures of the european union member states: the past and the future. Sustainability. 2022;14(2):1024. http://doi.org/10.3390/su14021024
    » http://doi.org/10.3390/su14021024
  • 20
    Chaboyer W, Harbeck E, Lee BO, Grealish L. Missed nursing care: an overview of reviews. Kaohsiung J Med Sci. 2021;37(2):82-91. http://doi.org/10.1002/kjm2.12308
    » http://doi.org/10.1002/kjm2.12308
  • 21
    Siqueira LDC, Caliri MHL, Kalisch B, Dantas RAS. Cultural adaptation and internal consistency analysis of the MISSCARE Survey for use in Brazil. Rev Lat Am Enfermagem. 2013;21(2):610-7. http://doi.org/10.1590/S0104-11692013000200019 PMid:23797556.
    » http://doi.org/10.1590/S0104-11692013000200019
  • 22
    Nymark C, Göransson KE, Saboonchi F, Falk AC, Von Vogelsang AC. Translation, culture adaption and psychometric testing of the MISSCARE survey-Swedish version. J Clin Nurs. 2020;29(23-24):4645-52. http://doi.org/10.1111/jocn.15505 PMid:32956504.
    » http://doi.org/10.1111/jocn.15505
  • 23
    Lisby M, Tei R, Mainz H, Gregersen M, Andersen KV. A Danish version of the MISSCARE survey: translation and validation. Scand J Caring Sci. 2023;37(1):301-12. http://doi.org/10.1111/scs.13143 PMid:36598034.
    » http://doi.org/10.1111/scs.13143
  • 24
    Wild D, Grove A, Martin M, Eremenco S, McElroy S, Verjee-Lorenz A et al. Principles of good practice for the translation and cultural adaptation process for Patient-Reported Outcomes (PRO) measures: report of the ISPOR Task Force for Translation and Cultural Adaptation. Value Health. 2005;8(2):94-104. http://doi.org/10.1111/j.1524-4733.2005.04054.x PMid:15804318.
    » http://doi.org/10.1111/j.1524-4733.2005.04054.x
  • 25
    Carvajal A, Centeno C, Watson R, Martínez M, Sanz Rubiales Á. ¿Cómo validar un instrumento de medida de la salud? An Sist Sanit Navar. 2011;34(1):63-72. http://doi.org/10.4321/S1137-66272011000100007 PMid:21532647.
    » http://doi.org/10.4321/S1137-66272011000100007
  • 26
    Rivero-Núñez E, González-Cárdenas L, Bayarre-Vea H. Validación y adaptación lingüística-cultural del Instrumento World Health Organization Quality of Live Bref (WHOQOL) en mujeres de edad mediana. Rev Cubana Enferm [Internet]. 2016 [citado 2020 abr 29];32(4):126-39. Disponible en: http://scielo.sld.cu/scielo.php?script=sci_arttext&pid=S0864-03192016000400017&lng=es&tlng=es
    » http://scielo.sld.cu/scielo.php?script=sci_arttext&pid=S0864-03192016000400017&lng=es&tlng=es
  • 27
    Ramada-Rodilla JM, Serra-Pujadas C, Delclós-Clanchet GL. Adaptación cultural y validación de cuestionarios de salud: revisión y recomendaciones metodológicas. Salud Publica Mex. 2013;55(1):57-66. http://doi.org/10.1590/S0036-36342013000100009 PMid:23370259.
    » http://doi.org/10.1590/S0036-36342013000100009
  • 28
    Ferreira M, Haas V, Dantas R, Felix M, Galvão C. Cultural adaptation and validation of an instrument on barriers for the use of research results. Rev Lat Am Enfermagem. 2017;25(0):28-52. http://doi.org/10.1590/1518-8345.1652.2852
    » http://doi.org/10.1590/1518-8345.1652.2852

Edited by

Associated editor

Fábio da Costa Carbogim https://orcid.org/0000-0003-2065-5998

Scientific editor

Marcelle Miranda da Silva https://orcid.org/0000-0003-4872-7252

Publication Dates

  • Publication in this collection
    26 Aug 2024
  • Date of issue
    2024

History

  • Received
    26 Dec 2023
  • Accepted
    09 May 2024
Universidade Federal do Rio de Janeiro Rua Afonso Cavalcanti, 275, Cidade Nova, 20211-110 - Rio de Janeiro - RJ - Brasil, Tel: +55 21 3398-0952 e 3398-0941 - Rio de Janeiro - RJ - Brazil
E-mail: annaneryrevista@gmail.com