Construction of a tool to measure perceptions about the use of the World Health Organization Safe Surgery Checklist Program

Abstract

Background:

The World Health Organization (WHO) has recommended greater attention to patient safety, particularly regarding preventable adverse events. The Safe Surgery Saves Lives (CSSV) program was released recommending the application of a surgical checklist for items on the safety of procedures. The checklist implementation reduced the hospital mortality rate in the first 30 days. In Brazil, we found no studies of anesthesiologists’ adherence to the practice of the checklist.

Objective:

The main objective was to develop a tool to measure the attitude of anesthesiologists and residents regarding the use of checklist in the perioperative period.

Method:

This was a cross-sectional study performed during the 59th CBA in BH/MG, whose participants were enrolled physicians who responded to the questionnaire with quantitative epidemiological approach.

Results:

From the sample of 459 participants who answered the questionnaire, 55% were male, 44.2% under 10 years of practice, and 15.5% with over 30 years of medical school completion. Seven items with 78% reliability coefficient were selected. There was a statistically significant difference between the groups of anesthesiologists who reported using the instrument in less or more than 70% of patients, indicating that the attitude questionnaire discriminates between these two groups of professionals.

Conclusions:

The seven items questionnaire showed adequate internal consistency and a well-defined factor structure, and can be used as a tool to measure the anesthesiologists’ perceptions about the checklist usefulness and applicability.

KEYWORDS
Tool; Checklist; Safe surgery; Adverse events; World Health Organization

Resumo

Introdução:

A Organização Mundial da Saúde (OMS) tem recomendado uma maior atenção com a segurança do paciente, mais especificamente em relação aos eventos adversos evitáveis. Foi lançado o programa “Cirurgia Segura Salva Vidas (CSSV)”, que recomenda a aplicação da lista de verificação cirúrgica (checklist) para a conferência de itens relacionados à segurança do procedimento. A implantação do checklist reduziu a mortalidade hospitalar nos primeiros 30 dias. No Brasil, não foram identificados estudos sobre adesão dos anestesiologistas à prática do checklist.

Objetivo:

Desenvolvimento de uma ferramenta para mensuração da atitude dos anestesiologistas e residentes em relação ao uso do checklist no período perioperatório.

Método:

Estudo transversal feito durante o 59° Congresso Brasileiro de Anestesiologia (CBA), em Belo Horizonte (MG), cujos participantes foram médicos inscritos e que responderam ao questionário com abordagem epidemiológica quantitativa.

Resultados:

A amostra constou de 459 participantes que responderam ao questionário, 55% do sexo masculino, 44,2% com menos de 10 anos e 15,5% acima de 30 anos de conclusão do curso médico. Foram selecionados sete itens com coeficiente de confiabilidade de 78%. Houve diferença estatisticamente significativa entre os grupos de anestesiologistas que referiram usar o instrumento em menos ou mais de 70% dos pacientes assistidos. Isso indica que o questionário de atitudes discrimina entre esses dois grupos de profissionais.

Conclusões:

O questionário de sete itens mostrou adequada consistência interna e uma estrutura fatorial bem delimitada. Pode ser usado como ferramenta para medida das percepções de anestesiologistas quanto à utilidade e a aplicabilidade do checklist.

PALAVRAS-CHAVE
Ferramenta; Checklist; Cirurgia segura; Eventos adversos; Organização Mundial da Saúde

Introduction

The 55th World Health Assembly, World Health Organization (WHO), through the Resolution 55.18 has recommended to its member states increased attention to patient safety, particularly regarding preventable adverse events. So, in October 2005 it was introduced the first Global Patient Safety Challenge with the theme: Clean Care is Safer Care, while the second Global Challenge addressed the fundamentals and practices of surgical safety focusing on prevention of surgical site infections, safe anesthesia, surgical teams, and use of indicators for surgical care.

In 2009, the WHO launched the Safe Surgery Saves Lives (SSSL) program,¹ which recommends the application of the surgical safety checklist, which determines three breaks in the perioperative period for checking the items regarding procedure security. The checklist and its implementation manual were translated into Portuguese and published by the Ministry of Health/ANVISA/PAHO (Fig. 1).

Figure 1
Surgical safety checklist.

The method chosen² for the project “Safe Surgery” was based on the tool created at the Johns Hopkins Medical Institution for improved communication among professionals participating in the operating room team. The main objective was for all the operation planning to be shared and, thus, facilitate the integration in the procedure implementation.

This structured communication tool, inspired by aviation, fragments complex tasks in more watertight steps, in order to reduce the chances of forgetting a key item for the quality and safety of the whole care. Checking the items listed in a previously made checklist speeds the process and, at the same time, creates barriers to any process failures.³ One of the decisive factors for success is the understanding of its usefulness by all those who make up the teams involved and, perhaps, this is the greatest obstacle to the program implementation. The perception of all should be that it is not a control method, but a useful tool for reducing flaws in the process, as shown in a multicenter study coordinated by Haynes et al.⁴ The use of checklist has been stimulated by accrediting organizations operating in Brazil that consider it an instrument of perioperative best practices,⁵ but one cannot say that it has become a widespread and common practice. New procedures like this among health professionals are not always received positively at first, particularly when the effectiveness of its results, though peremptory, is not easily demonstrated in the short term.

Thus, adherence to checklist depends on having positive attitudes and perceptions regarding its usefulness and applicability, inserted, preferably, in institutions that emphasize safety culture, because it is believed to be an essential requirement to reduce the occurrence of adverse events affecting rates security and entail reduction of patients’ morbidity and mortality.^6-12 The implementation of the Safe Surgery Checklist Program reduced hospital mortality in the first 30 days. Although the impact on the outcome was lower than previously reported, the effect depends crucially on the adherence to the routine use of the checklist.¹²

In Brazil, we found no studies of anesthesiologists’ adherence to the WHO “Surgical Safety Checklist” nor reports on adherence to type of subgroups of health care institution, department of anesthesiology practice and procedure complexity. Thus, assessing the attitude of professionals regarding adherence to checklist is very important, particularly in the development of educational activities that encourage the safety culture, as anesthesiologists are present in all three stages of its implementation, as well as nursing staff. The checklist introduction in several European centers also faced barriers and biases and led to some difficulties in its implementation, but with the safety culture in anesthesia there was a better standard of acceptance and modification of the standard practice of such centers and adequacy for the checklist regular use.^12-14 The main objective of this study was the development of a tool to measure the attitude of anesthesiologists and residents regarding the use of the checklist in the perioperative period.

Method

Cross-sectional study performed during de 59th Brazilian Anesthesiology Congress in Belo Horizonte (MG), whose participants were enrolled anesthesiologists and residents who went to the stand of the Brazilian Society of Anesthesiology (SBA) and voluntarily responded to the web-based self-administered electronic questionnaire with quantitative epidemiological approach. The study was approved by the Research Ethics Committee of the National Institute of Cardiology, and written informed consent was obtained from all participants — anonymity was guaranteed.

The instrument design prioritized the inclusion of questions that could contribute to the understanding of perceptions and attitudes of the profession being studied. Responses were obtained on a 5-point Likert scale⁷ (strongly disagree, partially disagree, neither agree nor disagree, partially agree, and strongly agree) to measure attitudes of respondents on various aspects of the checklist applicability and usefulness. We accept as positive answers those in which participants marked options 4 and 5 (partially agree/strongly agree) to the sentences that have been formulated in a positive way, or 1 and 2 (strongly disagree/partially disagree) to the questions formulated negatively. In this process, the scores assigned to items with negative connotations (1 and 2 to the questions formulated positively and 4 and 5 to those formulated negatively) were reversed, so as to produce an instrument with mean maximum range equal to 5 points, representing the best attitudes and perceptions regarding the checklist.

The questionnaire reliability was evaluated through internal consistency, estimated by the Cronbach's alpha reliability coefficient, relevant because the questionnaire was applied once in the sample. Cronbach's alpha values range from 0 to 1 and the lower limit usually accepted is 0.60.⁸ After selecting the items using the correlation coefficient with total score ≥0.3, the Cronbach's alpha coefficient was calculated. The selected items were subjected to principal components factor analysis with orthogonal rotation (Varimax) to identify the factorial structure of the instrument.

Discriminant validity of the instrument scores was tested by calculating the difference between the factor and total scores of the questionnaire among anesthesiologists groups who reported using the checklist in less or more than 70% of patients (Table 1).

Table 1
Mean difference between total and factorial scores of questionnaire between anesthesiologists groups using the checklist in less or more than 70% of patients.

	Mean difference	95% Mean difference CI		p
		Inferior	Superior
F1	−0.31	−0.42	−0.20	0.00
F2	−0.15	−0.29	−0.02	0.03
Total	−0.25	−0.35	−0.14	0.00

Predictive validity was assessed (a) using the calculation of Spearman's correlation coefficients between the scores and the binary variable of checklist use in less or more than 70% of cases (the outcome variable) and (b) the analysis of the ROC curve parameters between the total score and the outcome variable.

Results

The total sample of 459 participants who completed the questionnaire in four days of the aforementioned Congress, 55% male and 44.2% with less than 10 years and 15.5% over 30 years of medical school completion. Only 2.2% said they have done or be doing a specialization course or residency in anesthesiology. The average age of participants was 40.7 years.

Seven items with 78% reliability coefficient were selected. Factor analysis identified two factors: F1, related to the perception of the utility, and F2 related to the perception of the applicability of the checklist (Table 2). These two factors explained 58% of the variance in scores.

Table 2
Perceived utility factors and applicability.

Questions regarding the perception factor on utility (F1)

• Are you familiar with the “Safe Surgery Saves Lives” project (Checklist)?

• It is useful and prevents errors during surgery

• It is useful and prevents errors during anesthesia

• Checklist should be mandatory

Questions regarding the perception factor on applicability (F2)

• Should be used only in institutions with Hospital Accreditation

• Should be applied solely by nurses

• Checklist is not applicable to invasive procedures in the Diagnostic Center, such as spinal infiltration

There was a statistically significant difference between the groups of anesthesiologists who reported using the instrument in less or more than 70% of patients. This indicates that the attitude questionnaire discriminates between these two groups of professionals.

The correlation coefficients between the factor and total scores and the outcome variable were rho = 0.32, p < 0.01 for F1 scores; rho = 0.14, p < 0.02 for F2 scores; and rho = 0.28, p < 0.01 for total score. These values are significant, but the predictive validity of the questionnaire is low. These findings are substantiated by the percentage of area under the curve = 0.66 (0.61-0.71). This indicates low sensitivity and specificity of the instrument as a predictor of the use or non-use of the checklist in more than 70% of patients seen by anesthesiologists.

Discussion

The WHO Safe Surgery Saves Lives program aims to increase the quality and safety standards in health care; contemplate the prevention of sentinel events, surgical site infections, safe anesthesia, safe surgical teams, and indicators of surgical care. It was decided that the instrument would serve a core set of safety standards that could be applied globally and in different scenarios. In this set, it was obvious the multidisciplinary character, including all who work and contribute to the excellence of patient care.

Checklists are quick and simple tools that should be used by all professionals, from beginners to experts. It is not like an “instruction manual” or even the same as the standard operating procedures (SOP). Checklists, as proposed in the WHO “Safe Surgery Saves Lives” project for application before induction of anesthesia, before skin incision and before the patient leaves the operating room, should be brief, but without compromising the effectiveness regarding the objectives to which it is proposed.

The use of checklist is being monitored by the WHO,⁹ and currently about 1800 health care facilities report their systematic use, particularly in the countries of North America and Europe, in addition to an increasing implementation in Asia and Middle East. Brazil, so far, does not appear as one of the countries that express the use of checklist to its fullest.

The SBA presented the WHO “Safe Surgery Saves Lives” project to Brazilian anesthesiologists for the first time in 2009¹⁰ and, since then, the Quality and Safety Commission in Anesthesia (CQSA) has been promoting the project dissemination actions in all official events of the Society and many of its regional facilities nationwide. However, there was no way to have an estimate of anesthesiologists who work in the country and use the checklist. Thus, the Board of SBA in 2012 accepted the proposal of CQSA to perform a survey during the 59th CBA in BH/MG and provided resources for the survey application presented here.

The choice of the event was due primarily to allow that the largest number of participants could respond to the questionnaire, as the number of subscribers, around 2500 professionals, allow it to reach a high percentage of response, which is essential for successful studies using questionnaires as a tool, as well as being a more homogeneous sample due to the participation of professionals from around the country, even though the sample does not include anesthetists who do not usually attend the Congress of Anesthesiology.

Conclusions

The 7-item questionnaire showed adequate internal consistency (Cronbach's alpha coefficient > 0.7) and a well-defined factorial structure, it can be used as a tool to measure anesthesiologists’ perceptions about the usefulness and applicability of the WHO Safe Surgery Saves Lives checklist. However, although the perceptions captured by the questionnaire are able to discriminate among anesthesiologists with highest and lowest percentage of use checklist in practice, the scores resulting from the questionnaires cannot predict the use of the instrument. That is, despite the more or less positive perceptions about the checklist, there is no association between the probability of using or not the instrument in practice.

References

• ¹
  Organização Mundial da Saúde. Segundo desafio global para a segurança do paciente: Cirurgias seguras salvam vidas (orientações para cirurgia segura da OMS). Rio de Janeiro: Organização Pan-Americana da Saúde, Ministério da Saúde, Agência Nacional de Vigilância Sanitária; 2009.
• ²
  Makary MA, Holzmueller CG, Thompson D, et al. Operating room briefings: working on the same page. Jt Comm J Qual Patient Saf. 2006;32:351-5.
• ³
  Manser T, Foster S, Flin R, et al. Team communication during patient handover from the operating room: more than facts and figures. Hum Factors. 2013;55:138-56.
• ⁴
  Haynes AB, Weiser TG, Berry WR, et al. A surgical safety checklist to reduce morbidity and mortality in a global population. N Engl J Med. 2009;360:491-9.
• ⁵
  Joint Commission International. https://manual.jointcommission.org/Manual/WebHome [accessed 02.08.14].
  » https://manual.jointcommission.org/Manual/WebHome
• ⁶
  Handler SM, Castle NG, Studentski AS, et al. Patient safety culture assessment in the nursing home. Qual Saf Health Care. 2006;15:400-4.
• ⁷
  Wolfle D, Likert R, Marquis DG, et al. Standards for appraising psychological research. Am Psychol. 1949;4:320-8.
• ⁸
  Hair JF, Black WC, Babin BJ, et al. Análise Multivariada de Dados. 6ª ed. Porto Alegre: Bookman; 2009.
• ⁹
  Surgical Safety Web Map. http://maps.cga.harvard.edu:8080/Hospital/ [accessed 22.09.13].
  » http://maps.cga.harvard.edu:8080/Hospital/
• ¹⁰
  Correa RT. Guia da Organização Mundial da Saúde para a segurança do ato anestésico-cirúrgico. Anestesia em Revista. 2009;2:16-7.
• ¹¹
  Mastracci TM. What are the effects of introducing the WHO “surgical safety checklist” on in-hospital mortality? J Am Coll Surg. 2013;217:1151-3.
• ¹²
  Russ SJ. A qualitative evaluation of the barriers and facilitators toward implementation of the WHO surgical safety checklist across hospitals in England: lessons from the “Surgical Checklist Implementation Project”. Ann Surg. 2015;261:81-91.
• ¹³
  World Health Organization. Safe surgery saves lives: second global patient safety challenge; 2008. Available at: http://www.who.int/patientsafety/safesurgery/knowledgebase/SSSLBrochurefinalJun08.pdf [accessed 13.04.14].
  » http://www.who.int/patientsafety/safesurgery/knowledgebase/SSSLBrochurefinalJun08.pdf
• ¹⁴
  Soria-Aledo V. Dificultades en la implantacion del check list en los quiro fanos de cirugía. Cir Esp. 2012;90:180-185.

Publication Dates

• Publication in this collection
  Jul-Aug 2016

History

• Received
  2 Sept 2014
• Accepted
  11 Nov 2014