Patient safety in Intensive Care Units: development of a research project

ABSTRACT

Objective

: To describe our experience in the many processes involved in the development of a Project on Research into Intensive Care Unit Patient Safety.

Method

: Mixed design study: historic cohort study of the collection of data on patients and on adverse events/incidents and transversal design on the collection of data on a nursing team. The data were collected over a period of 90 days in 2012 at the Instituto Central do Hospital das Clínicas da Faculdade de Medicina of the Universidade de São Paulo (ICHC-FMUSP)and the University Hospital of the Universidade de São Paulo HU-USP). Procedures carried out: This study involved a number of stages: application of the Nursing Activities Score (NAS) at the ICHC-FMUSP, creation of a database system, hospital record inputs, monitor training, patient data extraction and load, collection of data during duty shift changes, and records.

Final considerations

: Training, researcher commitment, and collaboration with IT (Information Technology) professionals were crucial to the quality of the results obtained and of scientific production achieved. We hope that our report will serve to guide and encourage researchers to carry out complex surveys contributing to improve nursing and health knowledge.

Nursing, Team; Intensive Care Units; Patient Safety; Workload; Nursing Research

RESUMO

Objetivo

Relatar a experiência sobre os diferentes processos envolvidos no desenvolvimento de um Projeto de Pesquisa em Segurança do Paciente em Unidades de Terapia Intensiva.

Método

Estudo com delineamento misto: coorte histórica para a coleta dos dados dos pacientes e eventos adversos/incidentes e transversal para a coleta dos dados da equipe de enfermagem. A coleta de dados ocorreu durante 90 dias, em 2012, no Instituto Central do Hospital das Clínicas da Universidade de São Paulo e o Hospital Universitário da Universidade de São Paulo.

Processos desenvolvidos

A pesquisa envolveu diversas etapas para sua efetivação: implantação do Nursing Activities Score (NAS) no Instituto Central do Hospital das Clínicas da Universidade de São Paulo, desenvolvimento de sistema de banco de dados, digitalização de prontuários, treinamento de monitores, extração e carga de dados dos pacientes e coleta de dados durante a passagem de plantão, prontuários.

Considerações finais

Treinamentos, comprometimento dos pesquisadores e parceria com profissionais da tecnologia da informação foram fundamentais para a qualidade dos resultados obtidos e da produção científica alcançada. Espera-se que esse relato de experiência possa orientar e encorajar os pesquisadores a realizar pesquisas complexas que contribuam para a construção do conhecimento na enfermagem e saúde.

Equipe de Enfermagem; Unidades de Terapia Intensiva; Segurança do Paciente; Carga de Trabalho; Pesquisa em Enfermagem

RESUMEN

Objetivo

Presentar la experiencia acerca de los diferentes procesos involucrados en el desarrollo de un Proyecto de Investigación en Seguridad del Paciente en las Unidad de Cuidados Intensivos.

Método

Estudio con diseño mixto: cohorte histórica para la recolección de datos de pacientes y eventos adversos/incidentes y transversal para la recolección de datos del personal de enfermería. Se realizó la recolección de dados durante 90 días, en 2012, en el Instituto Central de Hospital de Clínicas de la Universidad de São Paulo y el Hospital Universitario de la Universidad de São Paulo, fueron los campos de estudio, donde.

Desarrollo de los procesos

La investigación incluyó varios pasos para la realización: implantación del Nursing Activities Score (NAS) en el Instituto Central de Hospital de Clínicas de la Universidad de São Paulo, desarrollo del sistema del banco de datos, digitalización de los registros, capacitación de los recolectores, extracción de datos de los pacientes y datos recolectados por medio de escalas, cambio de turno e historias clínicas.

Consideraciones finales

Formación, compromiso de los investigadores y la sociedad con profesionales de la tecnología fueron la clave para la calidad de los resultados obtenidos. Se espera que la descripción de esta experiencia pueda guiar los investigadores para realizar investigaciones complejas que contribuyen a la construcción del conocimiento en enfermería y salud.

Grupo de Enfermería; Unidades de Cuidados Intensivos; Seguridad del Paciente; Carga de Trabajo; Investigación en Enfermería

INTRODUCTION

Patient safety has become increasingly important since the disclosure of the report, “To Err is Human”, published in 2000 by the Institute of Medicine (IOM). This resulted in two studies assessing the incidence of adverse events (AE) carried out in two US hospitals and which exposed a high recurrence rate for such events in the health service. These results showed cases of AE occurrences in 2.9% to 3.7% of care given and also revealed the occurrence of 44,000 to 98,000 preventable deaths per annum, arising from unsafe acts, and which have led to concern on a global scale¹.

The World Health Organization (WHO) defines patient safety as “The reduction and mitigation of unsafe acts within the health-care system, as well as through the use of best practices shown to lead top optimal patient outcomes”². To explain and broaden comprehension in the context of patient safety, we present the following definition of incidents: incidents involving harm, also known as adverse events (AE) are those that do not arise from the natural evolution of the base illness, can be incapacitating, lead to increases in hospitalization time and cost, and to mortality. In addition to incidents with harm (AE), the WHO also defines incidents without harm (I) such as those that do not lead to measurable lesions or prolonged hospitalization².

The hospital unit in which the majority of severely ill patients are cared for, i.e., the Intensive Care Unit (ICU), is where most Adverse Events/Incidents (AE/I) occur. This is because such patients require complex and intensive care, and are more susceptible to errors in nursing care³.

Literature in this field has shown several nursing care factors with the potential to contribute to AE occurrences. Among these are nursing work load, stress, burnout, level of professional satisfaction, features of the nursing practice environment, and the safety culture of the respective institution⁴-⁶.

In this respect, based on the findings of a research project that identified clinical patient factors, work demands, and human factors involving the nursing team in AE/I cases, we submit this article with one aim: to report our experience in the many procedures involved in a Research Project on Patient Safety in an Intensive Care Unit (ICU).

METHOD

This is a report on the experience of researchers and graduate and post-graduate students of the Escola de Enfermagem (School of Nursing) at the Universidade de São Paulo, enrolled in the Programa de Pós-Graduação em Enfermagem na Saúde do Adulto (PPGAHNS - Graduate Program of Adult Health Nursing), and of the professional nursing staff of eight ICUs in a high complexity university hospital, and of an ICU and a Semi-Intensive Unit (SIU) of a secondary level university hospital.

The Patient safety in intensive care units: influence of human nursing factors on the occurrence of adverse events study was financed by the Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP – Research Support Foundation of the State of São Paulo) (protocols No. 11/51874-5 and 13/22671-4) and the Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq – National Science and Technology Development Council) Universal Project (No. 477860/2010-9), Coordenação para o Aperfeiçoamento de Pessoal de Ensino Superior (CAPES - Coordination for the improvement of Higher Education Personnel (CAPES) and Fundación Mapfre. The overall aim of this study was to investigate the association between patients’ demographic and clinical data, nursing workload, stress levels, prevalence of burnout and coping factors, professional satisfaction of the nursing staff, perception of the nursing practices environment, and the safety culture within the critical units and AE/I occurrences.

This was a mixed delineation study, with an historical cohort for patient and AE/I data collection and transversal design for data collection of the nursing professionals. It was carried out in eight ICUs of the Instituto Central do Hospital das Clínicas da Universidade de São Paulo (ICHC-USP), a tertiary teaching hospital and in the ICU and the SIU of the USP University Hospital (HU-USP), a secondary teaching hospital, both based in the municipality of São Paulo.

The project was approved by the Ethics in Research Committee of both institutions, Case No. 0196/2011, CAPPesq ICHC-FMUSP, and No. 1086/10, CEP HU/USP.

Patient data were collected over 90 consecutive days, from May 3 through July 31, 2012, at HU-USP, and from September 3 to December 1, 2012, at the ICHC-USP, by researchers and a group of monitors trained to collect data from hospital records, while another group monitored duty shift changes. At both institutions, the convenience sample included all patients aged 18 and over in the ICUs, regardless of their length of stay. The difference in the data collection period was due to the need to adapt the physical and administrative infrastructures for carrying out the project; these differed from one institution to the other. It is important to note that the ICHC-USP did not apply the Nursing Activities Score (NAS) to measure the nursing workload, unlike the HU-USP, where it has been in place since 2003.

During the first stage of the study, patient data collected included socio-demographic and clinical background. Among these were medical diagnosis, type of treatment (surgery or clinical), length of stay in the Unit, ICU and hospital discharge conditions, comorbidity, and the overall severity of each patient’s clinical condition. The patients’ comorbidity study was based on the Charlson Index⁷ and the severity of illness on the Simplified Acute Physiology II and the Logistic Organ Dysfunction System (LODS) scores⁹.

The analysis of the AE/I data, obtained from the hospital records, was based on the taxonomy suggested by the World Health Organization (WHO), which classifies AE/I by type in: Clinical Administration, Clinical process/Procedures, Documentation, Healthcare associated Infection, medication/fluids, Bood/ Blood Products, Nutrition, Oxygen/ Gases /Vapors, Medical device/equipment, Behavior, Patient Accidents, Infrastructure/ Building/ Premises, Resources/Organizational management. The data were also classified in accordance with the categories: Pathophysiology, Injury and Other. The AE degree of harm was classified as mild, moderate, severe and death².

The second stage of the investigation, which concentrates on a study of human nursing factors, evaluated the nursing team. This involved bio-social and working data, stress levels, burnout, professional satisfaction, the nursing practice environments of these units, and the institution’s safety culture. The nursing staff sample consisted of all the nurses, nursing technicians, and nursing aides who provided care throughout the duration of the study, excluding any who were on temporary leave for any reason.

The measurement instruments utilized during this stage were applied throughout the entire months of June and October 2012, respectively, at the HU-USP and ICHC-USP. The tools included: Nursing Activities Score (NAS)¹⁰, Workplace Stress Scale (WSS)¹¹, Medical Symptoms and Signs of Stress (MSSS)¹², Maslach Burnout Inventory¹³, Index of Work Satisfaction (IWS)¹⁴, Nursing Work Index - Revised (NWI-R)¹⁵, and Hospital Survey on Patient Safety Culture (HSOPSC)¹⁶.

PROCESSES INVOLVED IN CREATING THE PROJECT

Carrying out this project included the preparation of several processes to meet the objectives of the study. One of these was to make available a system for measuring nursing workload in the ICUs for the institutions involved, in order to comply with current legislation governing the creating and activities of ICUs in Brazil¹⁷ and, also, to contribute to dimensioning nursing staff, all based on patient care needs.

From April 2011 through April 2012, the basic activities carried out related to an infrastructure enabling the success of the project and included the following processes: 1) nursing staff training, the development and introduction of the NAS computing system; 2) development of the research database system; 3) storing all record data; 4) training analysts/collaborators for AE/I analysis; 5) extraction, transformation, and load of the identification data, hospitalization, doctors’ prescriptions, and laboratory tests from the databases of the various hospitals, to the project’s own data base, known as the Universal Database (UDB); 6) training monitors to observe duty shift changes; 7) application of stress, burnout, and coping tools for the nursing team, professional satisfaction, perception of the practices environment, and the respective institution’s safety culture.

Process 1: Training of nurses, development and introduction of the NAS computing system

Preparing the nurses to collect NAS data, a variable to measuring nursing workload, converted into hours of nursing, required almost a year. This was because the process dealt with a measuring tool hitherto unknown to the majority of the ICHC-USP ICU.

NAS was developed by Dr. Dinis Reis Miranda and colleagues in 2003¹⁰, translated and validated to Brazilian culture¹⁸to record the time spent on direct and indirect nursing activities. This also included administrative and family support activities.

Each item presents a point and the total sum represents the score given to a patient. This score corresponds to the respective patient’s direct and indirect nursing assistance needs during the preceding 24 hours¹⁸-²⁰.

This tool consists of seven categories and 23 items, with a variation of between 1,2 to 32 points. The Basic Activities category includes the following items: 1- monitoring and tritation, 2- laboratory, 3- medication, with the exception of vasoactive drugs, 4- hygiene procedures, 5- care of drains, 6- mobilization and positioning, 7- support and care of relatives and patients, 8- administrative and management tasks. The Ventilatory Support category includes items 9, 10, and 11, the Cardiovascular Support category, items 12, 13, 14, and 15, Renal support, items 16 and 17, Neurological support, item 18, Metabolic support, items 19, 2, and 21, and Specific interventions, items 22 and 23. It should also be noted that items 1, 4, 6, 7, and 8 include sub-items enabling the assessment of nursing care in accordance with the level of demand presented by the respective patient¹⁰.

The maximum NAS score is 176.8%; it is expressed as a percentage to indicate the proportionate time devoted by the nursing team to caring for the patient. For example, a patient presenting a score of 100% is one who required 100% of a nursing professional’s care over the preceding 24 hours. This same professional would be unable to care for another patient during this time as this could undermine safety and quality¹⁹.

Theoretical and practical training courses for implementing the NAS were given to the nurses. These consisted of morning (0800h) and evening (1900h) classes over a two-week period in June 2011.

Upon completion of this stage, supervised intensive training to consolidate the application of this tool, was given by two researchers, in loco, in the eight ICHC-USP ICU units from July 2011 through January 2012. It is important to note that an NAS manual, created by the researchers, was made available in each unit, for verification purposes in the event of doubt. In February 2012, NAS compliance tests were carried out. These were collected by nurses and specialists (gold standard) via application of the tool to 116 patients over three consecutive days. The result of the comparison test of averages (paired t-test) showed a significant (p=0.016), but low (0.224), correlation, where the difference between the averages was 8.9% (p=0.000). The agreement assessment between the observers showed little correlation in sub-items 1,4,6,7, and 8 and in item 9. In other words, the nurses underestimated the NAS scoring in relation to the gold standard.

Based on these results, further training was given with emphasis on specific items. Subsequently, the NAS was input into the computer system by a nursing professional holding a doctorate, a member of the Programa Nacional de Pós-Doutorado – PNPD (National Post-Doctorate Program) of the CAPES, jointly with IT professionals from the Institution, who were contracted for this specific project. The end purpose of developing this system was to leave it as a product/legacy for the institution in order to meet the needs of the research partnership between the EEUSP and the Divisão de Enfermagem (Nursing Division) at ICHC-USP and also to comply with government requirements that ICUs utilize a tool to measure nursing staff workload.

The development of a computer system consumed more time than originally estimated. This was due to problem management, which included the absence of an infrastructure for installing the computers in the ICUs and communication problems with the institution’s professionals, who did not understand the utilization of the system as being an ICU management care tool, far beyond of the objectives of a scientific investigation. This fact underscores the researchers’ commitment to the complete and satisfactory implementation of NAS as an ICU routine with the ICHC-USP Nursing Division. Eight computers were also acquired and installed in the ICUs and, at the conclusion of the project, donated to the institution.

This process was more streamlined in the HU-USP ICUs and SIUs, since the institution already had its own NAS computer system in place, and it had been applied in their Units since 2003. In these units, the project activity consisted solely of updating the nursing staff on the use of the tool, and training them to use it, in March and April 2012.

Process 2: Developing the database system

The study’s data collection included the patients’ socio-demographic information, laboratory standard data, therapeutic devices, medical prescriptions and AE/I in the UCUs. To prioritize the process and avert data manipulation errors, a computer system had to be developed. IT specialists developed a system involving an SQL (Structured Query Language) database or, more specifically, SQL Server 2008. The system was implemented in C# (C-Sharp) using DotNet 4.0 technology, and was named Universal Database (UDB) and allocated to an EEUSP database server in August 2012.

The system developed to collect the AE/I includes a friendly interface and installation via internet, with automatic update of versions. In other words, when alterations were made to the system, upon logging in, the system automatically updated the version installed in the analyst’s computer. Each analyst was first registered with his/her own login and password. The system also has a record of all operations carried out. The AE/I classification was carried out in line with the WHO proposal, since it was in a universal and standard language. This decision required the researchers to carry out consensual discussions and to prepare a standard data collection manual.

Process 3: Digital input of records

The problems encountered in accessing records for AE/I analysis required the introduction of a specific strategy involving inputting these records. This method, applied in the two institutions, required the purchase of portable scanners, a selection of digital monitors, check of material produced, and the development of a system to generate a single file per individual record. This was because, for each input page, the equipment generated a PDF format file, which facilitated reading and guaranteed organized records. The file name of each digitalized record was the name of the respective patient.

After the institutions had agreed on this activity, the digital inputting individuals were trained and an algorithm prepared specifying the documents to be input. Please note that this strategy brought major contributions to the project, since it enabled the analysts to access the PDF records in their computers, at times of their own choosing. This eliminated the need to physically move around the city and to use the institution’s premises, in addition to minimizing potential record errors.

The records were distributed among the analysts according to each one’s availability. At first, this distribution occurred via DVD and pen drive copies and, subsequently via Dropbox and Google Drive, the two latter enabling faster distribution and sharing of records. When one analyst had completed his/her batch, another analyst’s batch could immediately be accessed. This also enabled group discussions, some of them over long distances, and accelerated the processes required to reach an agreement.

The chief problems arose due to the absence of certain records which were held by other services or were on loan to other users, in addition to the need for re-inputting the digital work arising from difficulties in deciphering handwriting.

Process 4: AE/I assessment training for the analysts

As stated above, AE/I identification and classification in this study were carried out along WHO proposed lines. For record data collection, all 15 analysts were trained to read the data and manipulate the system. To standardize AE/I identification and classification among the analysts, a manual on the chief situations, potential AE/I, and periodic discussions on findings was prepared.

Each analyst received a set of records duly input in PDF format, including hospitalization addenda, ICU evolution, nursing staff notes, nursing staff prescriptions, and inter-consultations. This analysis commenced in mid-February 2013 and was continued for almost 18 months, in both institutions, involving a total of 806 hospitalizations in the ICHC-USP and 382 in the HU-USP.

One of the problems encountered in this process was keeping the group of analysts, all of them experienced clinical nurses, fully capable of making appropriate judgments to identify and classify AE/I. This is due to the extensive volume of detailed data reading and the immense number of variables to be collected, and which led to exhaustion and caused many of the analysts to abandon the work. This then led to the need to identify and train new participants.

Process 5: Extraction, transformation, and load of patient identification, hospitalization, laboratory testing, and medical prescription data

Given the high volume of ICU patient data and, in order to prevent the errors inherent to manual information transcriptions, identification, hospitalization, and laboratory test data were directly extracted from the hospitals’ databases. We were only able to extract the respective doctors’ prescriptions from the ICHC-USP database, since the HU-USP did not have electronic prescriptions. In addition to our collaborative work with these hospitals’ IT department, the volume in the database was obtained with the assistance of two professional Business Intelligence members ETL (Extract, Transform, and Load) experience. Data checks and cleaning, focusing solely on ICU patients were performed by the researchers themselves.

Process 6: Collection of adverse events/incidents during duty shift changes

AE/I collections during duty shift changes were shown as 10% of the 90 days of duty shift changes in relation to days, period, and ICU. This resulted in a total of 390 physically witnessed duty shift changes at the ICHC-USP and 89 at the HU-USP.

The monitors received full training in respect of time, location, and recording of all information received from the nursing team, in specific printed documents. Based on these records, the researchers identified the AE/I, in accordance with WHO classification.

The data were analyzed and checked by a third researcher and, subsequently, reconciled against the data obtained by the hospital record analysts. This process was carried out in compliance with one of the project objectives, i.e., checking that the AE/I advised in the duty shift changeovers were documented in the respective patients’ records.

Process 7: Scales of measurement of stress, coping, and burnout tools of the nursing staff, professional satisfaction, perception of the practices environment, and the respective institution’s safety culture

In order to comply with the objectives of all the projects involved, eight questionnaires were given to the ICHC and HU nursing teams. This process was carried out by the researchers in June 2012 at HU-USP and in October of the same year at ICHC-USP.

The nursing staff professionals responded to this tool in the appropriate location in the unit itself, during their working hours. The groups were organized by the head nurses, at a time of lowest volume of activity in the unit, and were monitored by one of the researchers in charge, who distributed these tools, explained the aim of the research, guaranteed the confidentiality and anonymity of the information collected, in addition to clarifying the risks and benefits of the survey, by obtaining these individuals’ free and informed consent. The average time incurred to complete these questionnaires was one hour. After the collection, the data were input in Excel and processed by grantee PNPD. At the ICHC-USP, the questionnaires were answered by 287 (83.4%) nursing staff out of a total of 344 who met the study inclusion criteria; no individuals were absent for vacation, leave, or other reasons. At the HU-USP, the return was 50 (75.7%) of the completed questionnaire, out of a total number of 66 nursing staff professionals.

FINAL CONSIDERATIONS

Carrying out complex and wide-ranging research projects is no easy task. Accordingly, the aim of these findings experienced and covered in this article is to encourage other nursing and health area researchers to carry out similar studies.

The various stages of data preparation and the actual process of gathering the information required the involvement of all the researchers to bring this project to life. Here, we highlight the groundwork of the researchers, now, in the first year of the development of the study, in respect of the overall comprehension of the specific methodologies involved. This is because the project’s dimension and complexity demanded skills beyond routine research abilities. For this reason, we created a project management course for the entire research team (coordinator, post-doctorate students, doctorate students, undergraduate students), to introduce them to feasible management methodologies to create the study which ensured satisfactory management of all stages of the project. The technical support provided by a management professional played a vital role in the study; it greatly aided the group to understand the subject matter and contributed significantly to the training of our researchers, particularly, the doctorate and post-doctorate students.

The end result of the project included the work of a post-doctorate student, three doctorate theses, a master’s degree dissertation, and three scientific preliminary studies by some undergraduate students. In addition to the volume of data and information available, other studies continue to be developed and will result in, at least, one more doctoral thesis, one master’s degree dissertation, and one scientific initiation study. Additionally, investigations into specific topics relating to patient safety, such as kidney infections and complications were carried out by PPGAHNS professors and presented at scientific events in Brazil and overseas; they are currently at the publication stage.

Further to the NAS system, the vital legacy left to the ICHC-USP continues to the present day. It also exists in a website version which enabled its use by the Orthopedic Institute (OI) and by the Psychiatry Unit (PI) of the HC-USP, and is planned to become available also at the Instituto do Coração-USP - INCOR-USP (USP Heart Institute). To date, over 80 thousand NAS measurements were taken and are now available to the nursing managements of the institutions. These will contribute significantly to analyzing and calculating the need for nursing human resources for the ICUs.

The entire process of preparing, planning, and creating a project of this magnitude was a huge challenge for the researchers and students involved. Each step, each procedure with its attendant complexities and limitations, was a demanding apprenticeship and shared responsibility. The harmony and commitment of each individual involved were crucial to finalizing the data collection, analyzing the results, and preparing the definitive research reports.

A further important aspect is to underscore the partnership entered into with the many professionals involved, particularly, the IT experts, who became key collaborators in this nursing survey.

It is our hope that this report on our study and the description of its many processes will guide and encourage other researchers to carry out complex surveys, thereby contributing to the advancement of our knowledge of nursing and health issues.

REFERÊNCIAS

• ¹
  Kohn LT, Corrigan JM, Donaldson MS, McKay T, Pike KC. To err is human. Washington, DC: National Academy Press; 2000.
• ²
  World Health Organization (WHO). Patient Safety. The conceptual framework for the International Classification for Patient Safety v1.1. Final technical report and technical annexes [Internet]. Geneva; 2009 [cited 2015 Jan 23]. Available from: http://www.who.int/patientsafety/implementation/taxonomy/publications/en/
  » http://www.who.int/patientsafety/implementation/taxonomy/publications/en/
• ³
  Beccaria LM, Pereira RAM, Contrin LM, Lobo SMA, Trajano DHL. Eventos adversos na assistência de enfermagem em uma unidade de terapia intensiva. Rev Bras Ter Intensiva. 2009;21(3):276-82.
• ⁴
  Holden RJ, Scanlon MC, Patel NR, Kaushal R, Kamisha HE, Brown RL, et al. A human factors framework and study of the effect of nursing workload on patient safety and employee quality of working life. Quality Health Care. 2011;20(1):15-24.
• ⁵
  Aiken LH, Sermeus W, Heede KVH, Sloane DM, Busse R, McKee M, et al. Patient safety, satisfaction, and quality of hospital care: cross sectional surveys of nurses and patients in 12 countries in Europe and the United States. BMJ. 2012;344:e1717.
• ⁶
  Daud-Gallotti RM, Costa SF, Guimarães T, Padilha KG, Inoue EN, Vasconcelos TN, et al. Nursing workload as a risck for healthcare associated infections in ICU: a prospective study. PLoS One. 2012:7(12):e52342.
• ⁷
  Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40(5):373-83.
• ⁸
  Le Gall JR, Lemeshow S, Saulnier F. A new simplified acute physiology score (SAPS II) based on a European/North American multicenter study. JAMA. 1993;270(24): 2957-63.
• ⁹
  Le Gall JR, Klar J, Lemeshow S, Saulnier F, Alberti C, Artigas A, et al. The Logistic Organ Dysfunction system. A new way to assess organ dysfunction in the intensive care unit. JAMA. 1996;276(10):802-10.
• ¹⁰
  Miranda DR, Raoul N, Rijk A, Schaufeli W, Iapichino G. Nursing activities score. Crit Care Med. 2003;31(2):374-82.
• ¹¹
  Paschoal T, Tamayo A. Validação da escala de estresse no trabalho. Estud Psicol. 2004;9(1):45-52.
• ¹²
  Ferreira EAG, Marques AP, Matsutani LA, Vasconcellos EG, Mendonça LLF. Avaliação da dor e estresse em pacientes com fibromialgia. Rev Bras Reumatol. 2002;42(2):104-10.
• ¹³
  Maslach C, Jackson S, Leiter M. Maslach inventory manual. Palo Alto: Consulting Psychologists; 1986.
• ¹⁴
  Stamps PL, Piedmonte EB. Nurses and work satisfaction: an index for measurement. Am J Nurs. 1986;98(3):16KK-16LL.
• ¹⁵
  Aiken LH, Patrician PA. Measuring organizational traits of hospitals: the revise nursing work index. Nurs Res 2000;49(3):146-53.
• ¹⁶
  Reis CT, Laguardia J, Martins M. Adaptação transcultural da versão brasileira do Hospital Surveyon Patient Safety Culture: etapa inicial. Cad Saúde Pública. 2012; 28(11):2199-210.
• ¹⁷
  Brasil. Ministério da Saúde; Agência de Vigilância Sanitária. Resolução n. 7, de 24 de fevereiro de 2010. Dispõe sobre os requisitos mínimos para funcionamento de Unidades de Terapia Intensiva e dá outras providências [Internet]. Brasília; 2010 [citado 2015 jan. 23]. Disponível em: http://bvsms.saude.gov.br/bvs/saudelegis/anvi/2010/res0007_24_02_2010.html
  » http://bvsms.saude.gov.br/bvs/saudelegis/anvi/2010/res0007_24_02_2010.html
• ¹⁸
  Queijo AF, Padilha KG. Nursing Activities Score (NAS): croos-cultural adaptation and validation to portuguese language. Rev Esc Enferm USP [Internet]. 2009 [cited 2015 Jan 23];43(n.spe):1001-8. Available from: http://www.scielo.br/pdf/reeusp/v43nspe/en_a04v43ns.pdf
  » http://www.scielo.br/pdf/reeusp/v43nspe/en_a04v43ns.pdf
• ¹⁹
  Conishi RMY; Gaidzinski RR. Nursing Activities Score (NAS) como instrumento para medir carga de trabalho de enfermagem em UTI adulto. Rev Esc Enferm USP [Internet]. 2007 [citado 2015 jan. 23];41(3):346-54. Disponível em: http://www.ee.usp.br/reeusp/upload/pdf/725.pdf
  » http://www.ee.usp.br/reeusp/upload/pdf/725.pdf
• ²⁰
  Padilha KG, Sousa RMC, Queijo AF, Mendes AM, Miranda DR. Nursing Activities Score in the intensive care unit: Analysis of the related factors. Intensive Crit Care Nurs. 2008;24(3):197-204.

• Financial Support: Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES), Fundação de Amparo à Pesquisa de São Paulo (FAPESP), Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq), and Fundación Mapfre.

Publication Dates

• Publication in this collection
  Dec 2015

History

• Received
  23 Apr 2015
• Accepted
  05 Aug 2015