Adaptation and validation of the World Health Organization’s on Safe Childbirth Checklist for the Brazilian context

Abstract

Objectives:

to culturally adapt and validate the WHO Safe Childbirth Checklist (SCC) in Brazilian hospitals.

Methods:

a methodological study was carried out with consensus techniques and cross-cultural adaptation stages. The original SCC underwent three adaptation and validation stages: 1- nominal group with a panel of experts; 2- consensus conference at two maternity schools, in meetings with professionals who would use the list; 3- pre-test with a structured questionnaire for health professionals from both maternities (n=40) after 30 days of using the checklist. Validation criteria contemplated the content validity, adequated to Brazilian protocols, terminology and feasibility for local context.

Results:

the adapted SCC in Brazil was called the Lista de Verificação para o Parto Seguro - Brasil (LVPS-BR) (Checklist for Safe Childbirth -Brazil) and included 49 items. In the first stage, the 29 items of the original SCC were approved with some adaptations (e.g. CD4 was replaced by the Rapid HIV Test). In the second stage, some of the 29 items were adjusted and added 24 items more. In the third stage, three items were excluded, two were grouped and one more was added.

Conclusions:

the validation process provided a potentially useful LVPS for the Brazilian context, presenting validity and feasibility evidences for the Brazilian context.

Key words
Healthcare quality; Patient’s safety; Maternal-child health services; Checklist

Resumo

Objetivos:

adaptar culturalmente e validar o Safe Childbirth Checklist (SCC) da OMS para os hospitais brasileiros.

Métodos:

realizou-se uma pesquisa metodológica com técnicas de consenso e etapas de adaptação transcultural. O SCC original passou por três etapas de adaptação e validação: 1- grupo nominal com painel de especialistas; 2- conferência de consenso em duas maternidades escolas, em reuniões com profissionais que utilizariam a lista; 3- pré-teste com questionário estruturado aos profissionais de saúde das duas maternidades (n=40) após 30 dias de sua utilização. Critérios de validação contemplaram as validades de face e conteúdo, adequação aos protocolos nacionais, terminologia e viabilidade no contexto local.

Resultados:

o SCC adaptado para o Brasil foi denominado Lista de Verificação para o Parto Seguro - Brasil (LVPS-BR), contendo 49 itens. Na primeira etapa, os 29 itens do SCC original foram aprovados com algumas adaptações (ex.: CD4 substituído por Teste Rápido para HIV). Na segunda etapa, ocorreram ajustes em alguns dos 29 itens e acrescentaram-se 24 itens. Na terceira etapa, excluíram-se três itens, agruparam-se dois e acrescentou-se um.

Conclusões:

o processo de validação disponibilizou uma LVPS potencialmente útil para o contexto brasileiro, apresentando indícios de validade e viabilidade para o contexto nacional.

Palavras-chave
Qualidade dos cuidados de saúde; Segurança do paciente; Serviços de saúde Materno-infantil; Lista de verificação

Introduction

Mother-child mortality is still a serious public health problem in Brazil and worldwide. Considering that most of these deaths occur by preventable causes, these deaths may be indicators of the quality in the health system.¹

In developing regions are responsible for 99% of maternal deaths. In Latin America and the Caribbean, the Maternal Mortality Rate (MMR) report 85 deaths per 100,000 live births (LB), this is considered high when compared to Central Asian countries with 39 deaths/100,000 LB and North Africa with 69 deaths/100,000 LB.²

During 2000-2011, the MMR in Brazil was 64.8 deaths/100,000 LB with a neonatal mortality rate (NMR) of 10.6 deaths/1,000 LB.³ Despite the declining trend in the maternal mortality in the last decade, the MMR in Brazil was far from the proposed goal by signatory countries of the Millennium Development Goals (MDG) in 2015, which was 35 maternal deaths/100,000 LB in 2015.⁴

On the other hand, child mortality is considered one of the best indicators to assess health care quality, as well as the socioeconomic level of a population.^5,6

Brazil reached the goal of reducing childhood mortality (children under 5 years old) by 53.7 deaths in 1990 to 17.7 in 2011, due to the health programs implemented in the 1980s, 1990s and 2000s, such as encouraging breastfeeding, oral dehydration, immunizations, Estratégia de Saúde da Família (ESF) (Family Health Strategy) and social welfare program as Bolsa Família (Family Allowance). However, further reductions in child mortality have been difficult to achieve (referring to the first year of life), more specifically in the neonatal period, when approximately 60% of the child deaths occur, in which could be prevented with a good health care quality.^7,4

As 99% of the childbirths in Brazil are performed in a health unit and 91% receive prenatal care coverage⁴ this can be inferred that not only the access to health services ensure sufficient good results for women and children.^8,9 The CEDAW (Committee on the Elimination of Discrimination Against Women) monitoring report¹⁰ points out the difficulties in providing quality of care for women in relation to professionals’ training that attend in the prenatal and childbirth units; access to health services; the non-based evidence technology use; and a quality of care that is concerned with patient’s safety and the integrality in care.

With the purpose of improving the quality and ensuring safe childbirth, the World Health Organization (WHO) developed the Safe Childbirth Checklist (SCC)¹¹ in 2008 to enable maternity hospitals to provide standardized care in relation to the main risks associated to childbirth.

The document as originally developed in English by a group of specialists, as a universal action, and contained 29 easy-to-apply items that required the use of a few resources and it was easily applied in the first world hospitals or in developing countries in order to ensure adverse-free and irreversible damage in childbirth care, such as maternal or neonatal deaths. The checklist is a simple but effective instrument that not only helps to ensure the implementation on important health care processes, as well as helps to improve direct communication and coordination among the team members.¹²

This relates to the main causes of maternal (hemorrhage, infection, obstructed childbirth labor, and hypertensive disorders)¹³ and newborns’ deaths (inadequate intra-delivery care, perinatal asphyxiation, infection and premature birth complications).¹⁴ The SCC was tested in ten African and Asian countries obtaining positive results.¹⁵

A study conducted in India by Harvard School of Public Health (HSPH) with the Safe Childbirth Checklist revealed that the number of essential practices performed by the employees at a hospital increased an average of 10 to 25 of the 29 items after the checklist was implemented.¹⁶ A pilot study carried out in Colombia demonstrated that this instrument contributed to remind the health professionals to implement essential practices, such as washing their hands, controlling infection, assessing postpartum bleeding and breastfeeding up to one hour after birth, in addition to other important care.¹⁷

Is the SCC a feasible instrument for Brazilian reality? The WHO recognizes the need for cross-cultural adaptation and validation for local realities, in order to make it suitable for different contexts, languages and practices. The Cross-cultural Adaptation methodologies instrument¹⁸ and the Consensus Techniques¹⁹ to assess health interventions can be used for this purpose.

The present study aimed to adapt and validate the Safe Childbirth Checklist in the Brazilian hospital context, so that health professionals can use the instrument to remember and provide the necessary care for mothers and newborns during childbirth labor and delivery and the puerperium. We hoped for a qualified care in which would impact the provided quality of care and consequently decrease adverse events, especially maternal and neonatal mortality.

Methods

This is a methodological research that adapted and validated the instruments by using a combination of consensus techniques and cross-cultural adaptation stages. This became as the reference method in assessing the validation on the health intervention by the specialists’ consensus techniques, such as the nominal group and consensus conference.¹⁹ The Cross-cultural Adaptation proposed by Beaton et al.,¹⁸ which included a revision with a committee of specialists in phase 4, conducted with a nominal group and consensus conference, in addition a pre-test with the implementation of the instrument and subsequent questionnaire on its validity with 40 professionals in phase 5. The research was approved by the Research Ethics Committee at the Hospital Universitário Onofre Lopes/UFRN the protocol number is: 1.186.809/2015.

The project is part of the multicenter study registered in the WHO’s “Safe Childbirth Checklist (SCC) Collaboration”, in which has priority to expand its viability in using the SCC in countries around the world. The study counts on the participation of four Latin American countries (Brazil, Mexico, Peru and Venezuela) and one from East Africa (Tanzania). Mexico promoted the initiative, and this research is coordinated by the National Public Health Institute in Mexico, which is in charge of the cooperation among the countries. However, each country is independent in conducting, financing and coordinating the project. In Brazil, the coordinated is performed by the Grupo de Pesquisa Melhoria da Qualidade em Serviços de Saúde (QualiSaúde) (Quality Improvement in Health Services) affiliated with the Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq) (National Council for Scientific and Technological Development) and certified by the Universidade Federal do Rio Grande do Norte.

Following the activity flow presented in Figure 1 through the collaboration network, WHO provided a Safe Childbirth Checklist (SCC) in European Portuguese In this language, the title of the instrument was the “Lista de Verificação para Partos Seguros” (Checklist for Safe Childbirth). Based on this LVPS and the original SCC, the research group in Brazil adapted the terms in order to make them more suitable for the Brazilian Portuguese, resulting in the production of the Checklist for Safe Childbirth- Brazil (LVPS-BR), which was used for analysis and adapted in three stages as described in Figure 1.

Figure 1
Flowchart of the LVPS adaptation and validation.

In Stage 1 was used a nominal group technique variation in which a committee of specialists conducted an analysis proposed in phase 4 by Beaton et al.¹⁸ This technique can be applied when participants obtained experienced on the presented topic and when the group is not very heterogeneous and the number of participants is less than 10. With this technique usually it can have good decision-making validity group, which favors constructive conflict resolution and depersonalizing confrontations caused by disagreements. It is divided into 6 phases: 1. Generating ideas in individual form; 2. Registering a list of individual ideas; 3. Explaining ideas; 4. Prioritizing individual ideas (preliminary vote); 5. Discussing preliminary vote; 6. Final vote.²⁰ The technique used consists of the following: 1. Present the SCC items; 2. Explain the items; 3. Round 1 on criteria based assessment by specialists (first vote in-person); 4. Discussion among specialists; 5. Round II on the adaptation and reassessment with specialists (second vote in-person); 6. Discussion among specialists; 7. Round III on the adaptation and reassessment with the specialists (long distance final vote). During the voting rounds I, II and III of Stage 1, the professionals assessed each one of the 29 LVPS items in terms of validity and viability criteria for the Brazilian context (Figure 1).

The group of specialists consisted of nine professionals: three from a University Maternity Hospital in the countryside of Rio Grande do Norte State; three from a State Maternity Hospital located in the capital city (Natal); and three from a University Maternity Hospital in Natal city. The maternities were chosen by representing different profiles probably found in Brazil and the selection of professionals were based on the representativeness of the professions involved in the childbirth care. As for the profession, three professionals were obstetric nurses, three obstetricians and three pediatricians.

The group was coordinated by a specialist in quality management in health services, an infectologist and an epidemiologist. Stage 1 took place at the Departamento de Saúde Coletiva (DSC) at the Universidade Federal do Rio Grande do Norte (UFRN) during 5 hours.

In the second validation stage, there were meetings in form of Consensus Conference (CC), which also complemented on a wide perspective analysis by specialists’ committee proposed in phase 4 by Beaton et al.¹⁸ This stage included physicians, scientists and other professionals in order to obtain a consensus regarding a specific topic.²¹ A Consensus Conference was held at each one of both maternities that applied the LVPS, that is, one in the countryside and the other in capital of Rio Grande do Norte (RN).

The maternity in the countryside is a teaching hospital for gynecology, obstetrics and pediatrics care, located in the town of Santa Cruz, RN, with a population of 36,477 inhabitants.²² The hospital provides reference services in mother-child care for the Trairi region and other towns in the State. At the hospital, approximately 3 childbirths are performed per day and 1,300 per year. It offers accommodation, a surgical center and delivery rooms, but does not have a maternal or neonatal Intensive Care Unit (ICU). The team consists of obstetricians, pediatricians, obstetric nurses, nursing assistants and among others.

The maternity located in Natal (with approximately 817,590 inhabitants) is characterized as a reference unit for high-risk childbirths in the health services units in the Rio Grande do Norte (RN).²² Thus, in this maternity besides attending a great demand in Natal city, it receives patients form different regions of the State, performing an average of 12 deliveries per day and approximately 4,300 per year. It has accommodation, a surgery center, delivery rooms, a maternal and neonatal ICU and a milk bank. There is a health team composed by 65 physicians (obstetricians, intensivists and neonatologists) and 117 nurses.

The consensus conferences were organized by the Núcleo de Segurança do Paciente (Patient’s Safety Nucleus) and managed by both maternities. The CC was attended by all kinds of childbirth professionals (physicians, nurses, nutritionists, physiotherapists, nutritionists, pharmacists and nursing technicians) and included an item-by-item presentation of the LVPS, applying the same validity and viability criteria used in Stage 1. One researcher presented an instrument to be adapted, another researcher mediated the professionals’ observations and another researcher was in charge to write down the the professionals’ agreements and disagreements. At this stage it was possible to elaborate two specific LVPS, one for each maternity, which were then aggregated in one for the final analysis in Stage 3 as follows.

With the adapted LVPS, which contemplated with the contributions of the previous stages of the project, we conducted a pilot study with its introduction of the medical records of all the patients treated at these maternities, in which each stage of childbirth and puerperium was applied for the mothers and newborns during 30 days. To demonstrate how important to fill out the LVPS was, posters and pamphlets were produced and announcements were made on local television stations to disclose the project. The only difference between the instruments implemented at the two maternities is that one of the LVPS was divided into two instruments: one content of the items was related to the parturient /puerperium woman and the other with the item related to newborns. This occurred because the professionals at the service suggested that this would be easier to complete the LVPS, since at this maternity, the mothers and the babies’ medical records are separated.

A 30-day pre-test implementation of the LVPS was conducted in February 2016. After the implementation, a representative of the Núcleo de Segurança do Paciente (Patient’s Safety Nucleus) from each maternity applied a structured questionnaire to 40 professionals who filled out the LVPS, 20 from each maternity participant of the project. The pre-test and the quantity of professionals selected to fill out the questionnaire was based on the cross-cultural adaptation model proposed by Beaton et al.¹⁸ The instrument contained the same validity and viability criteria that were applied in Stage 1 and served to assess each item of the adapted list. The professional categories who answered the questionnaires consisted of 20 physicians and 20 nurses, who were included if their names were on at least five of the LVPS at the location where it was applied. The results of this stage served to consensus researchers and professionals from the maternities for the final version of the LVPS.

Five criteria were used to assess the validity and viability of the LVPS for the Brazilian context during the three stages of its adaptation and validation. The criteria are related to specific items that assess the content validity of the LVPS, adequated to the Brazilian protocols, terminology and viability in the local context.

In criterion 1 was on validity or the degree of obvious relevance,²⁰ and was asked whether the LVPS item “was relevant”. In criterion 2, was on the content of validity or whether the concept was measured meant to be actually measured,²⁰ and was asked if the item “was related to the mother and/or the baby’s safety during childbirth.” In criterion 3 was checked if the “item was adequate for the recommendations and national or local protocols”. In criterion 4 was asked if “the terminology is adequate” and in criterion 5 dealt with viability, by asking whether “is possible to measure in the context of a working environment”.

The options to answer each question of the criteria were on a Likert scale, ranging from: 1- I totally disagree; 2- I disagree; 3- Indifferent; 4- I agree and 5- I totally agree. The criteria were assessed in Stage 1 individually by each specialist from the Nominal Group; in Stage 2 by the consensus conference groups, and in Stage 3 individually by each professional who answered the questionnaire.

The validity was analyzed quantitatively in Stages 1 and 3 of the study since they produced individual scores for the validity and viability criteria. This analysis was based on the Índice de Validade (IV) (Validity Index) for each item on the list.²³

The IV was calculated in Stage 1 based on the median and in Stage 3 based on the 10^th and 20^th percentiles. This change in the parameter was justified in Stage 1since the level of demand for the consensus was lower, thus, they were widely recognized items and was approved by the WHO. In Stage 3, the level of demand had to be higher since there were additional items included in Stage 2 of the study.

The cutoff point to exclude an item from the final LVPS was a score of <4 in the IV. In Stage 3, however, some additional factors were considered for exclusion. The items with some criterion or IV <4 in the 10^th percentile had its assessed score in the 20th percentile if they were from the SCC. If the SCC item in the 20^th percentile was excluded, the researchers would analyze whether a cultural factor influenced the exclusion, despite it was indicated.

Results

After the first adaptation and validation stage, the SCC was denominated as Safe Childbirth Checklist (LVPS). After the three stages, validity and viability were approved with 49 items, 10 at the moment of admission, 9 before the delivery, 14 immediately after childbirth and 16 at hospital discharge. The adaptation process involved changes in the content, terminology, and adding and removing items, as shown in Table 1.

Table 1
Description of alterations during the adaptation phases.

LVPS assessed questions during the consensus phases
At the moment of admission
1. Does the pregnant woman need to be referred?	Stage 1: The term pregnant woman was changed to parturient. Stage 3: Added: referred to another hospital.
2. Was the partogram initiated?	Stage 1: The temperature registered was every 4 hours rather than every 2 hours. Stage 2: Does not apply was added to the check and register the temperature every 6 hours and register the blood pressure every 2 hours when Magnesium Sulfate was taken.
3. Does the pregnant woman need to take an antibiotic?	Stage 1: Administrate antibiotic when fever is unexplained. Stage 2: Left only membrane rupture > 18 hours and added another reason in the checklist, the item "no need for clinical and/or laboratorial reassessment" was added.
4. Does the pregnant woman need to take magnesium sulfate?	Stage 2: The reasons to administrate were: severe, pure or pre-eclampsia in conjunction with hypertension; DBP ≥110mmHg and/or clinical symptoms; headache, visual disorders and altered awareness level; epigastric pain, right upper quadrant pain ; nausea and vomiting and exalted patellar reflexes (increased amplitude and/or area obtained).
5. Does the pregnant woman need to take antiretroviral ?	Stage 1 : Unknown diagnosis was removed from the checklist and the parameters to check were changed from CD4 cell counts to confirmed seropositivity in the prenatal or after the rapid test. Stage 2: Does not apply to check was added. Stage 3: Does not apply was removed and the retroviral administration criterion became only seropositivity confirmed.
6. Were there hand-washing material and gloves for each vaginal examination?	Stage 1: Yes and no were added to be checked. Stage 2: Does not apply was added and modified the term hand washing was replaced by hand sanitation. Stage 3: Does not apply was replaced by material to check (water, soap, paper towel, alcohol solution and gloves).
7. Was the presence of a companion during delivery encouraged?	No changes were made.
8. Will the pregnant woman or companion ask for help during labor, if necessary? A. Did the woman bring her prenatal card? B. Does the parturient need to take an antihypertensive?	Stage 3: Added the alert signal decreasing fetal movements. Stage 2: Added. Stage 2: Added.
Before the vaginal delivery
9. Does the pregnant woman need to take an antibiotic?	Stage 1: In considering antibiotic administration mother's temperature ≥38oC was replaced by unexplained fever during labor. Stage 2: Left only membrane rupture >18hours and other reason was added. The item does not need clinical and/or laboratorial reassessment was added.
10. Does the pregnant woman need to take magnesium sulfate?	Stage 2: The reasons to administrate were: severe, pure or pre-eclampsia in conjunction with hypertension; DBP ≥110mmHg and/or clinical symptoms; headache, visual disorders and altered awareness level; epigastric pain, right upper quadrant pain; nausea and vomiting and exalted patellar reflexes (increased amplitude and/or area obtained).
11. Was there essential material near the bed and preparation for the delivery. For the pregnant woman	Stage 2: Two Kelly forceps were added to check.
12. Was there essential material near the bed and preparation for the delivery. For the newborn	Stage 1: The items to check were clean towels, sterile blade to cut the umbilical cord, suction device as well as a pediatric balloon and mask. Stage 2: Number 6, 8 and 10 tracheal probe and number 6 and 8 short gastric probes; meconium aspiration device; vacuum aspiration and gauge; manual neonatal resuscitator/self-inflating balloon; 00, 0 and 1 ventilation masks; wrist pulse oximeter; laryngoscope with straight blades number 00, 0 and 1; tracheal intubation tubes number 2.5/3/3.5/4; adrenalin; volume expander (0.9% saline solution or Ringer lactate); sterile fields; number 6 or 8 tracheal probe or 5F or 8F umbilical catheter; gloves and glasses, sterile blade to cut the umbilical cord; umbilical cord clamp; sources of oxygen/compressed air; radiant heat source; and a clock.
13. Was the assistant identified and ready to help during the delivery if necessary? C. Does the parturient need to have a cesarean? D. Does the parturient need to take an antihypertensive? E. Does the professional have recent updated neonatal resuscitation qualifications (maximum 2 years)? F. Does the parturient need to have an episiotomy?	Stage 1 : The question was changed to was the professional assistant identified and informed to be ready to provide delivery assistance if necessary?" and no and yes were added to be checked. Stage 3: Was a second professional identified and informed to assist with the delivery, if necessary? Stage 2: Added. Stage 2: Added. Stage 2: Added. Stage 3: Added.
After the delivery
14. Is the mother bleeding more than expected?	Stage 2: In the instructions was added: rectal misoprostol and call into action the rapid response team for obstetric emergencies.
15. Does the mother need to initiate antibiotics?	Stage 2: The administration criteria were changed to excessive manipulation in the delivery, forceps and other reasons. An item to the checklist was added: no, clinical and/or laboratory reassessment needed. Stage 3: Cesarean.
16. Does the mother need to initiate magnesium sulfate?	Stage 2: The reasons to administrate were changed to: Severe, pure or pre-eclampsia in conjunction with hypertension; DBP ≥110mmHg and/or clinical symptoms; headache, visual disorders and altered awareness level; epigastric pain, right upper quadrant pain; nausea and vomiting and exalted patellar reflexes (increased amplitude and/or area obtained).
17. Does the newborn need to be referred?	Stage 2: The question was changed to: does the newborn need to be referred to a special care unit? and the items in the checklist were no, yes, at the own institution or yes, to another institution". Stage 3: The question was changed to: does the newborn need to be referred to another hospital and the items in the checklist was no or yes, is/was provided.
18. Does the newborn need to take antibiotics?	Stage 2: In the administration criteria membrane rupture > 18 hours" and other reasons were added. And in the checklist another item was added no, clinical and/or laboratory reassessment needed.
19. Does the newborn need special care/monitoring?	Stage 2: The question was changed to special care or surveillance? And the criteria added were need to take antibiotic, needed resuscitation and other reasons.
20. Does the newborn need to initiate antiretroviral therapy?	Stage 2: Antiretroviral prophylaxis criterion changed from 12 hours after birth to 4 hours after birth.
21.1 Was there skin-to-skin contact (if the mother and the newborn are well)?	Stage 2: Item was divided.
21.2 Was breastfeeding initiated in the first hour (if the mother and the newborn are well)?	Stage 2: Item was divided.
22. Will the mother/companion ask for help if there are any signs of danger? G. Does the parturient need to take an antihypertensive? H. Was the cord clamped between 1 and 3 minutes? I. Was vitamin K administrated? J. Was prophylaxis for neonatal conjunctivitis performed?	Stage 2: There was less than normal in signs of danger activity. And the term signs of danger was replaced for signs of warning. Stage 2: Added. Stage 2: Added. Stage 2: Added. Stage 2: Added. Stage 3: Was not approved.
K. Did the NB have an identification bracelet on?	Stage 2: Added.
Before hospital discharge
23. Is the mother's bleeding controlled?	Was not modified.
24. Does the mother need to take an antibiotic?	Stage 2: In the administration criteria chills and foul smelling discharge were replaced by suspected endometritis and other reason. And the checklist an item was added no, need clinical and/or laboratorial reassessment.
25. Does the newborn need to take an antibiotic?	Stage 2: The question was modified to: if the newborn was taking antibiotics, was the treatment concluded?
26. Does the baby breastfeed correctly?	Stage 2: Good teaching practice on breastfeeding and postpone hospital discharge were not added to the checklist.
27. If the mother is seropositive, did the mother and the newborn receive sufficient amount of antiretrovirals (ARV) for the period of 6 weeks?	Stage 2: The checklist items were: - Yes, for the baby; - Yes, for the mother; - Yes, for the mother and the baby. Stage 3: Does not apply was added to the checklist.
28. Were family planning options discussed and presented to the mother?	Stage 3: Was not approved.
29.1 Was the follow-up organized and was it confirmed that the mother and/or companion will seek help if there are signs of danger for the mother?	Stage 2: It was divided in two questions: was the mother's follow-up organized and agreed to and was it confirmed that the mother and/or companion were informed of the signs of danger? The term signs of danger was replaced for signs of warning. Stage 3: The item was combined.
29.2 Was the follow-up organized and was it confirmed that the mother and/or companion will seek help if there are any signs of danger for the newborn?	Stage 2: It was divided in two questions: was the newborn's follow-up organized and agreed to and was it confirmed that the mother and/or companion were informed of the signs of danger? The term sings of danger was replaced for signs of warning. Stage 3: The item was combined.
L. Did the NB present signs of jaundice? M. Did the newborn perform blood group and RH factor tests?	Stage 2: Added. Stage 2: Added.
N. Did the newborn take BCG vaccine? O. Did the newborn take the hepatitis B vaccine? P.Was the neonatal heel prick test performed? Q. Was the newborn hearing screening performed? R. Was the red reflex examination performed? S. Was the tongue screening test performed? T. Was the screening for critical congenital heart defects test performed?	Stage 2: Added. Stage 2: Added. Stage 2: Added. Stage 2: Added. Stage 2: Added. Stage 2: Added. Stage 2: Added.

During Stage 1, considering all 29 items of the original SCC were approved in terms of validity criteria in the third round (Median IV >4). In this stage, five items were adapted due to the terminology and six items in relation to adequate content, considering national or local protocols or viability in the work context (Table 2).

Table 2
Content adaptations (≅) in terminology (T), new items (+) and eliminated items (x) performed during the three adaptation and validation stages.

Initial proposals of assessed questions during the consensus phases	Staqes			Final
	1	2	3
At the moment of admission
1. Does the pregnant woman need to be referred?	T		T	WHO
2. Was the partogram initiated?	≅	≅		≅ WHO
3. Does the pregnant woman need to take an antibiotic?	≅	≅		≅ WHO
4. Does the pregnant woman need to take magnesium sulfate?		≅		≅ WHO
5. Does the pregnant woman need to take an antiretroviral ?	≅	≅	≅	≅ WHO
6. Were there availability of hand-washing material and gloves for each vaginal examination?	≅	≅	≅	≅ WHO
7. Was the presence of a companion during the delivery encouraged?	≅			WHO
8. Will the pregnant woman or companion ask for help during labor if necessary?		≅		≅ WHO
A. Did the woman bring her prenatal card?		+		BR
B. Does the pregnant woman need to take an antihypertensive?		+		BR
Before the vaginal delivery
9. Does the pregnant woman need to take an antibiotic?	≅	≅		≅ WHO
10. Does the parturient need to take magnesium sulfate?		≅		≅ WHO
11. Was there essential material near the bed and preparation for the delivery confirmed. For the pregnant woman		≅		WHO
12. Was there essential material near the bed and preparation for the delivery confirmed. For the newborn		≅	≅	≅ WHO
13. Was the assistant identified and ready to help during the delivery if necessary?	T	T	T	WHO
C. Does the parturient show signs to have a cesarean?		+	≅	BR
D. Does the parturient need to take an antihypertensive?		+		BR
E. Is the current professional with recent updated neonatal resuscitation qualifications (maximum 2 years)?		+		BR
F. Does the parturient show signs to have an episiotomy?			+	BR
After childbirth
14. Is the mother bleeding more than expected?	T	≅		≅ WHO
15. Does the mother need to initiate antibiotics?	T	≅	≅	≅ WHO
16. Does the mother need to initiate magnesium sulfate?		≅		≅ WHO
17. Does the newborn need to be referred?	T	T	T	WHO
18. Does the newborn need antibiotics?		≅	≅	≅ WHO
19. Does the newborn need special care/monitoring?		≅	≅	≅ WHO
20. Does the newborn need to initiate antiretroviral therapy?		≅		≅ WHO
21.1 Was there skin-to-skin contact (if the mother and the newborn are well)?		≅		≅ WHO
21.2 Was breastfeeding initiated in the first hour (if the mother and the newborn are well)?		≅		≅ WHO
22. Will the mother/companion ask for help if there are any signs of danger?		≅	≅	≅ WHO
G. Does the parturient need to take an antihypertensive?		+		BR
H. Was the cord clamped between 1 and 3 minutes?		+		BR
I. Was vitamin K administrated?		+		BR
J. Was prophylaxis for neonatal conjunctivitis performed?		+	X	-
K. Did the NB have an identification bracelet on?		+		BR
L. Did the mother receive immediate care after childbirth?		+	X	BR
Before hospital discharge
23. Is the mother's bleeding controlled?				WHO
24. Does the mother need to take an antibiotic?		≅		≅ WHO
25. Does the newborn need to take an antibiotic?		≅	≅	≅ WHO
26. Does the baby breastfeed correctly?		≅		WHO
27. If the mother is seropositive, did the mother and the newborn receive sufficient amount of antiretrovirals (ARV) for a period of 6 weeks?		≅	≅	≅ WHO
28. Were family planning options discussed with the mother?			X	-
29.1 Was follow-up organized and confirmed that the mother and/or companion will seek help if there are signs of danger for the mother?		≅		≅ WHO
29.2 Was the mother's follow-up organized and agreed to?		+	X
29.3 Was the follow-up organized and confirmed that the mother and/or companion will seek help if there are any signs of danger for the newborn?		≅		≅ WHO
29.4 Was the newborn's follow-up organized and agreed to?		+	X
M. Did the NB show any signs of jaundice?		+		BR
N. Did the newborn perform blood group and RH factor tests?		+		BR
O. Did the newborn receive BCG vaccine?		+		BR
P. Did the newborn receive hepatitis B vaccine?		+		BR
Q. Was the neonatal heel prick test performed?		+		BR
R. Was the newborn hearing screening performed?		+		BR
S. Was the red reflex examination performed?		+		BR
T. Was the tongue screening test performed?		+		BR
U. Was the screening for critical congenital heart defects performed?		+		BR

• - Items indicated by cardinal numbers (1,2,3 ...) are those present on the original Safe Childbirth Checklist and items in letter order (A, B, C...) are those added in the adapted Safe Childbirth Checklist.
  - Items with unmodified content are indicated in the "final" column as WHO, those content that had adaptations are WHO and the original items on the Brazilian list are as BR.

In relation to the terminology, for example, two main modifications were made: (1) the terms “pregnant woman” and “mother” were substituted for “parturient”, which refers to the moments before childbirth and “puerperium”, when it referred to the postpartum. (2) Immediately before the vaginal delivery or cesarean, in the item “assistant identified and informed to be ready to assist in the delivery if necessary?”, the term “assistant identified” was substituted for “professional identified”.

In regards to the adaptations of the content, three main changes were made: (1) the temperature taking frequency of the parturient at the moment of admission was altered from every two hours to every six hours; (2) in the item “does the parturient need to take an antibiotic?” present in three moments on the list, the indication to administer an antibiotic to the parturient when her temperature is above 38ºC was not considered sufficient to prescribe an antibiotic; (3) at the moment of admission, in the item “ does the pregnant woman need to take any antiretroviral?”, the SCC recommendation was to administrate antiretroviral based on the CD4 cell count, but we altered it to “Administrate antiretroviral if seropositivity is confirmed in the prenatal or after the rapid test”. The possibility of a parturient being admitted without HIV diagnosis is the delay in obtaining the CD4 result and the routine in Brazilian maternities to conduct the rapid test to confirm seropositivity were the justifications used for this adaptation.

During Stage 2, in the conference with professionals from the two participating maternities, 24 items were added, 2 items were adapted as for the terminology and 25 items as for the adequate content (Table 2). The LVPS totaled 53 items in this stage.

For the added items: at the moment of admission, “did the parturient bring her prenatal card?”; immediately before the vaginal delivery (or cesarean), “is the parturient indicated for cesarean?” and “does the current professional have a recent updated neonatal resuscitation qualification?”; immediately after childbirth, “was the cord clamped between 1 and 3 minutes?”, “was vitamin K administrated ?’ “was prophylaxis performed for neonatal conjunctivitis?”, and “does the newborn have an identification bracelet on?”; and just before hospital discharge, “does the newborn have jaundice?”, “did the newborn perform blood group and RH factor tests?”, “did the newborn take BCG vaccine?”, “the hepatitis B vaccine?”, “neonatal heel prick?”, “newborn hearing screening?”, “red reflex examination?”, “neonatal tongue screening test?” and “screening for critical congenital heart defects (CCHD)?”. The item “does the parturient need to take an antihypertensive” was added at the moment of admission, immediately before the vaginal delivery or cesarean and immediately after child-birth.

In respect to the viability or the possibility of being measured at a work context, two main adaptations were made: (1) immediately after birth, the item “Started breastfeeding and skin-to-skin contact” was divided into two items, since breastfeeding would not be indicated if the puerperal was infected with HIV virus; (2) just before hospital discharge, the item “Organize follow-up and ensure mother/companion will seek help in case there are signs of danger for either the mother or the newborn, after hospital discharge”, this was divided into four items to have the possibility to organize the follow-up and before hospital discharge give instructions to the mothers and the newborns. This was also justified since hospital discharge process is divided between the obstetrician and pediatrician, being impossible to check the mother and the baby at the same time.

During the third stage, after considering the professionals’ responses to the questionnaire on each item and the established decision criteria, 49 items remained in the final version.

According to the data presented in Table 3, show that 41 of the 53 items analyzed in terms of IV obtained a median of 4 in the 10th percentile. This means that 90% of the professionals approved the item with an adequate median. Of the twelve remaining items with a median of less than 4, we analyzed whether they were part of the SCC of the WHO. The two items that were not part of the SCC were excluded (“conducted prophylaxis of neonatal conjunctivitis?” and “confirmed care provided to mother immediately after birth”) and the other 10 items that were part of the SCC were analyzed in terms of the IV in the 20^th percentile. Eight of the items that derived from the SCC were approved with a median of 4 in the 20^th percentile, meaning that 80% of the professionals agreed that the item should remain. However, two items continued with scores less than 4 (“was the presence of a companion during the delivery encouraged?” and “were family planning options discussed with the mother?”). Although, these items were not approved in the quantitative parameter, they were analyzed to determine if they were not approved if it was due to a typical cultural standard in the Brazilian context. The final decision was to include the item “was the presence of a companion encouraged?”, considering a strong recommendation regarding the importance of a companion at the time of the delivery to ensure the quality and safety of the care provided. On the other hand, the item that discusses the opinions on family planning was excluded from the list, since in the context studied, this activity is performed in the primary care and not in the hospital setting.

Table 3
Validity analysis results of 53 items that are in the 5^th version of the LVPS during the 3^rd adaptation and validation stage and final decision on the last version.

Item	10th percentile (n=40)					Validity index P10	20th percentile (n=40)					Validity index P20	Final decision
	C1	C2	C3	C4	C5		C1	C2	C3	C4	C5
At the moment of admission
1	3	4	3	3	3	3	4	4	4	4	4	4	Included
2	4	4	3	4	4	4							Included
3	4	4	4	4	4	4							Included
4	4	4	4	4	4	4							Included
5	4	4	4	4	4	4							Included
6	2	3	3	3	3	3	4	4	4	4	4	4	Included
7	2	2	2	3	3	2	4	3	3	3	4	3*	Included
8	4	3	3	3	3	3	4	4	4	4	4	4	Included
A	4	4	4	4	4	4							Included
B	4	4	4	4	4	4							Included
Immediately before the vaginal delivery (or cesarean)
9	4	4	3	4	4	4							Included
10	4	4	4	4	4	4							Included
11	4	4	4	4	4	4							Included
12	4	4	4	4	4	4							Included
13	2	2	2	2	2	2	4	4	4	4	4	4	Included
C	3	3	3	3	3	3	4	4	4	4	4	4	Included
D	4	4	3	4	3	4							Included
E	4	4	4	4	4	4							Included
F													NA
Immediately after the delivery
14	4	4	4	4	4	4							Included
15	4	4	4	4	4	4							Included
16	4	4	4	4	4	4							Included
17	3	3	3	3	3	3	4	4	4	4	4	4	Included
18	4	4	3	4	4	4							Included
19	4	4	4	4	4	4							Included
20	4	4	4	4	4	4							Included
21.1	3	3	3	3	3	3	4	4	4	4	4	4	Included
21.2	3	3	3	3	3	3	4	4	4	4	4	4	Included
22	4	4	4	4	4	4							Included
G	4	4	4	4	4	4							Included
H	4	4	4	4	4	4							Included
I	4	4	4	4	4	4							Included
J	2	2	1	2	3	2	3	3	2	3	3	3	Excluded
K	4	4	4	4	4	4							Included
L	4	3	3	3	3	3	4	4	3	3	3	3	Excluded
Before hospital discharge
23	4	4	4	4	4	4							Included
24	4	4	4	4	4	4							Included
25	4	4	4	4	4	4							Included
26	4	4	4	4	4	4							Included
27	4	4	4	4	4	4							Included
28	3	2	2	3	2	2	4	3	3	3	4	3	Excluded
29.1	4	4	4	4	4	4							Included
29.2	4	4	2	4	4	4							Included
29.3	4	4	4	4	3	4							Included
29.4	4	3	4	3	4	4							Included
M	4	4	4	4	4	4							Included
N	4	4	4	4	4	4							Included
O	4	4	4	4	4	4							Included
P	4	4	4	4	4	4							Included
Q	4	4	4	4	4	4							Included
R	4	4	4	4	4	4							Included
S	4	4	4	4	4	4							Included
T	4	4	3	4	4	4							Included
U	4	4	3	4	4	4							Included

• Criteria: C1: validity; C2: validity of the content; C3: adequate protocols ; C4: terminology; C5: viability. Likert scale values: 1 – I totally disagree; 2- I disagree; 3- Indifferent; 4- I agree and 5- I totally agree
  Validity index: Median of C1 to C5 values in the 10th percentile obtained for all the items, and in the 20th percentile when the item was <4 in the 10th percentile and belonged to the SCC.
  NA: Does not apply, since it is a new item added to the consensus in the final stage.
• ^*
  Despite obtaining IV 3, the item was included after the researchers' analyzed qualitatively

Discussion

This study contributed to improve the quality of childbirth care in Brazil, once cross-cultural adaptation and assessing the validity and viability of a potentially useful instrument in preventing adverse outcomes, for both the mothers and the newborns. The translated list induces good practices that act directly (hypertensive diseases, hemorrhage and puerperal infection) and indirectly in causes of maternal death (circulatory system diseases related to pregnancy, childbirth or puerperium and prior infections)4 and neonatal deaths (inadequate intra-delivery care, perinatal asphyxia, infection and premature birth complications).^14,24

Childbirth care quality needs to be improved in Brazil and worldwide, since merely increasing the access to institutionalized childbirth is not enough to achieve good health care results in women and children. In addition to access to health services, the quality of services provided must also be guaranteed,⁸ a necessity which was evident in a recent study conducted in India.⁹ The LVPS adapted to Brazil intends to promote care quality with emphasis on patient’s safety or reducing risks and unnecessary harm associated to care to an acceptable minimum.^8,25

Many studies have proven that checklists are beneficial to improve the quality in health services and that they improve the adherence in good practices,^16,17,26 and may be used in resource both rich and poor regions.²⁵ This was confirmed in the study of a surgical checklist that influenced health indicators in high, medium and low income countries.²⁷ The LVPS was created with a global perspective to be used in countries with little diagnostic technology (such as clinical analyses and imaging laboratories), and for this reason, it is necessary to be adapted in the Brazilian context.¹¹

However, only to include the checklist in maternities will have no effect or validity without intervention to implement the adequate list. It is important that the LVPS should be adapted to meet the needs of the professionals and users and that communication and scientific evidence based on knowledge should be emphasized in the team. Furthermore, the continuous follow-up feedback is also essential to assure efficient, effective and safe care.^25,27

The validity of the LVPS also depends on the level of evidence of each of its items. Specialists have identified the need to update some of the items on the original list to the most updated protocols published in Brazil, such as the use of antiretroviral in the first 12 hours after birth, the use of the CD4 cell parameters to initiate treatment on HIV positive pregnant mothers, and to administrate parameters of magnesium sulfate. After the validation stages and indirect analysis of empirical evidence (criterion validity), the list was adapted according to the protocols.^28,29

As for the validity, after the pilot study at the two hospitals, all the items that remained in the final version obtained acceptable values in terms of the possibility of measuring them in a work context. This is consistent with professionals’ reports on the easy application of the instrument and the list was possible to be applied in the studied context. It was essential to apply the pre-test in order to identify and correct problems besides in reducing barriers to its use, a WHO recommendation for implementing the checklists.²⁵

In contrast to the Safe Surgery Checklist,²⁵ the LVPS encompasses the entire time that the woman spends at the maternity (admission, immediately before the delivery, immediately after the delivery and before hospital discharge) and two public target populations (mother and newborn). As such, to make its execution viable at one of the maternities, the list needed to be separated for the mother and the newborn, since the medical records at this hospital are separated and the professionals that assist them are different.

The validation process provided a Safe Childbirth Checklist that is potentially useful for the Brazilian context, indicating validity and viability. The adapted list was considered relevant, applicable to improve safety and suitable according to evidence based practices with adequate and viable terminology for different profiles in Brazilian hospitals, both in the countryside and the capital. However, its use must be preceded by assessment of the local context and adaptations, as recommended for all checklists and clinical protocols.

References

• ¹
  Habicht JP, Victora CG, Vaughan JP. Evaluation designs for adequacy, plausibility and probability of public health programme performance and impact. Int J Epidemiol. 1999; 28 (1): 10-8.
• ²
  WHO (World Health Organization). Trends in maternal mortality: 1990 to 2013: estimates by WHO, UNICEF, UNFPA, The World Bank and the United Nations Population Division. Geneva: 2014. [Acesso em: 20 de jan 2015]. Disponível em: http://apps.who.int/iris/bitstream/10665/112682/2/9789241507226_eng.pdf?ua=1
  » http://apps.who.int/iris/bitstream/10665/112682/2/9789241507226_eng.pdf?ua=1
• ³
  Brasil. Ministério da Saúde. Indicadores e dados básicos: indicadores de mortalidade. Brasil: 2012. [Acesso em: 29 mai 2015]. Disponível em: http://tabnet.datasus.gov.br/cgi/idb2012/matriz.htm#mort
  » http://tabnet.datasus.gov.br/cgi/idb2012/matriz.htm#mort
• ⁴
  Instituto de Pesquisa Econômica Aplicada e Secretaria de Planejamento e Investimentos Estratégicos. Objetivos de desenvolvimento do milênio: relatório nacional de acompanhamento. Brasil: [Acesso em: 29 mai 2015]. Disponível em: http://www.pnud.org.br/Docs/5_RelatorioNacionalAcompanhamentoODM.pdf
  » http://www.pnud.org.br/Docs/5_RelatorioNacionalAcompanhamentoODM.pdf
• ⁵
  Lourenço EC, Brunken GS, Luppi CG. Mortalidade infantil neonatal: estudo das causas evitáveis em Cuiabá, Mato Grosso, 2007. Epidemiol Serv Saúde. 2013; 22 (4): 697-706
• ⁶
  Graaf J, Steegers E, Bonsel G. Inequalities in perinatal and maternal health. Curr Opin Obstet Gynecol. 2013; 25 (2): 98-108.
• ⁷
  Victora CG, Aquino EM, do Carmo Leal M, Monteiro CA, Barros FC, Szwarcwald CL. Maternal and child health in Brazil: progress and challenges. Lancet. 2011; 377: 1863- 76.
• ⁸
  Scott KW, Jha AK. Putting quality on the global health agenda. N Engl J Med. 2014; 371 (1): 3-5. [Acesso em: 02 de março 2015]. Disponível em: http://www.nejm.org/doi/full/10.1056/NEJMp1402157#t=article
  » http://www.nejm.org/doi/full/10.1056/NEJMp1402157#t=article
• ⁹
  Randive B, Diwan V, Costa A. India's Conditional Cash Transfer Programme (the JSY) to Promote Institutional Birth: Is There an Association between Institutional Birth Proportion and Maternal Mortality? PLoS ONE. 2013; 8 (6): e67452.
• ¹⁰
  Consórcio Nacional de Redes e Organizações. Monitoramento da CEDAW: ação permanente do movimento de mulheres/Brasil. Porto Alegre: Coletivo Feminino Plural, 2014. Caderno 1 [Acesso em: 16 mar 2016]. Disponível em: http://monitoramentocedaw.com.br/wp-content/uploads/2013/08/caderno_cedaw.pdf
  » http://monitoramentocedaw.com.br/wp-content/uploads/2013/08/caderno_cedaw.pdf
• ¹¹
  WHO (World Health Organization). Safe Chilbirth Checklist. Geneva: 2015. [Acesso em: 16 mar 2016]. Disponível em: http://www.who.int/patientsafety/implementation/checklists/childbirth-checklist/en/
  » http://www.who.int/patientsafety/implementation/checklists/childbirth-checklist/en/
• ¹²
  Weiser TG, Berry WR. Review article: perioperative check-list methodologies. Can J Anaesth. 2013; 60 (2): 136-142.
• ¹³
  Van Den Broek NR, Graham WJ. Quality of care for maternal and newborn health: the neglected agenda. BJOG. 2009; 116 (1): 18-21.
• ¹⁴
  Lee AC, Cousens S, Darmstadt GL, Blencowe H, Pattinson R, Moran NF, Hofmeyr GJ, Haws RA, Bhutta SZ, Lawn JE. Care during labor and birth for the prevention of intrapartum-related neonatal deaths: a systematic review and delphi estimation of mortality effect. BMC Public Health. 2011; 11 (3): S10.
• ¹⁵
  WHO (World Health Organization). Safe childbirth check-list collaboration: improving the health of mothers and neonates progress report. Geneva; 2014 [Acesso em: 20 jun 2015] Disponível em: http://www.who.int/patientsafety/implementation/checklists/scc-progress-report-2014.pdf?ua=1
  » http://www.who.int/patientsafety/implementation/checklists/scc-progress-report-2014.pdf?ua=1
• ¹⁶
  Spector JM, et al. Improving quality of care for maternal and newborn health: prospective pilot study of the WHO safe childbirth checklist program. PLoS One. 2012; 7 (5): e35151.
• ¹⁷
  WHO (World Health Organization). Colombian nurses improve the quality of care at birth. Geneva; 2015 [Acesso em: 20 jun 2015]. Disponível em: http://www.who.int/features/2015/colombia-safe-childbirth-checklist/en/
  » http://www.who.int/features/2015/colombia-safe-childbirth-checklist/en/
• ¹⁸
  Beaton D, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine. 2000: 25 (24): 3186-91.
• ¹⁹
  Moreno SP, Argelaguete EP. No todo es acuerdo em consenso: limitaciones de los métodos de consenso em los servicios de salud. Gac Sanit. 1993; 7 (39): 294-301.
• ²⁰
  Saturno PJ, Antón JJ, Santiago MC. La construcción de criterios para evaluar la calidad: manual del máster em gestión de la calidad em los servicios de salud: módulo 3 - actividades básicas para la mejora continua: métodos y herramientas para la realización de ciclos de mejora. Universidad de Murcia. Murcia.
• ²¹
  Descritores em Ciências da Saúde. Conferência de Consenso. Brasil. [Acesso em: 9 jun 2015]. Disponível em: http://decs.bvs.br/
  » http://decs.bvs.br/
• ²²
  Brasil. Ministério da Saúde. Informações de saúde: dados demográficos e sócio-econômicos. Brasil. [Acesso em: 16 fev 2015]. Disponível em: http://tabnet.datasus.gov.br/cgi/deftohtm.exe?ibge/cnv/poprn.def
  » http://tabnet.datasus.gov.br/cgi/deftohtm.exe?ibge/cnv/poprn.def
• ²³
  Alexandre NMC, Coluci MZO. Validade de conteúdo nos processos de construção e adaptação de instrumentos de medidas. Ciênc Saúde Coletiva. 2011; 16 (7): 3061-8.
• ²⁴
  WHO (World Health Organization). Newborn care at birth. Geneva. [Acesso em: 28 jun 2016]. Disponível em: http://www.who.int/maternal_child_adolescent/topics/newborn/care_at_birth/en/
  » http://www.who.int/maternal_child_adolescent/topics/newborn/care_at_birth/en/
• ²⁵
  Organización Mundial de la Salud. Marco conceptual de la clasificación internacional para la seguridad del paciente. Genebra; 2009 [Acesso em: 20 jun 2015]. Disponível em: http://www.who.int/patientsafety/implementation/icps/icps_full_report_es.pdf
  » http://www.who.int/patientsafety/implementation/icps/icps_full_report_es.pdf
• ²⁶
  Patabendige M, Senanayake H. Implementation of the WHO safe childbirth checklist program at a tertiary care setting in Sri Lanka: a developing country experience. BMC Pregnancy Childbirth. 2015; (15): 12.
• ²⁷
  Gawande A. Checklist: Como fazer as coisas benfeitas. . Rio de Janeiro: Sextante; 2011.
• ²⁸
  Brasil. Ministério da Saúde. Secretaria de Vigilância em Saúde. Departamento de DST, Aids e Hepatites Virais. Guia de consulta rápida: protocolo clínico e diretrizes terapêuticas para manejo da infecção pelo HIV em crianças e adolescentes. Brasil: 2014 [Acesso em: 20 jun 2015]. Disponível em: http://www.aids.gov.br/sites/default/files/anexos/publicacao/2014/56319/_p_15_07_2014_protocolo_guia_rapido_pediatrico_p_25629.pdf
  » http://www.aids.gov.br/sites/default/files/anexos/publicacao/2014/56319/_p_15_07_2014_protocolo_guia_rapido_pediatrico_p_25629.pdf
• ²⁹
  Secretaria de Estado de Saúde Pública do Rio Grande do Norte. Protocolo de assistência materno infantil do Estado do Rio Grande do Norte. Brasil: 2015 [Acesso em: 27 out 2017]. Disponível em: http://adcon.rn.gov.br/ACERVO/sesap/DOC/DOC000000000080196.PDF
  » http://adcon.rn.gov.br/ACERVO/sesap/DOC/DOC000000000080196.PDF

Publication Dates

• Publication in this collection
  Apr-Jun 2018

History

• Received
  08 Feb 2017
• Accepted
  30 Mar 2018