Reliability and validity of the Brazilian version of the HSOPSC: a reassessment study

Confiabilidade e validade da versão brasileira do HSOPSC: um estudo de reavaliação

Fiabilidad y validez de la versión brasileña del HSOPSC: un estudio de revisión

Abstract:

This study aimed to reassess the psychometric properties of the Hospital Survey on Patient Safety Culture (HSOPSC) adapted to Portuguese for use in the Brazilian context. An observational, cross-sectional study was performed in a large, private, non-profit, acute care hospital, reference in patient safety, in a major Brazilian metropolis. Participants were selected from a non-probability sample of all eligible personnel in the various hospital departments invited to participate in the study. Reliability of the HSOPSC was assessed by estimating Cronbach’s alpha for each dimension. confirmatory factor analysis (CFA), a matrix of correlations between the dimensions, and exploratory structural equation modelling (ESEM) were used in exploratory and confirmatory analyses of the structural validity of the construct. The overall response rate was 18.7% (n = 1,439). Four dimensions (“overall perceptions of patient safety”; “staffing”; “teamwork across units”; and “non-punitive response to error”) returned problems of internal consistency. CFA returned acceptable fit with the original 12-dimension model. Correlations between the dimensions of the original 12-dimension model indicated discriminant validity problems, while residual variance was greater than 0.70 in 13 items. The ESEM of the original 12-dimension model returned good fit, with the following indices: CFI = 0.985; TLI = 0.968, and RMSEA = 0.026 (90%CI: 0.024-0.029). Although better than those of the first evaluation, the results obtained in this validity and reliability reassessment of the Brazilian version of the HSOPSC require further research.

Keywords:
Patient Safety; Organizational Culture; Health Care Surveys; Reproducibility of Results

Resumo:

O estudo teve como objetivo reavaliar as propriedades psicométricas da versão do Hospital Survey on Patient Safety Culture (HSOPSC) adaptada para o português, para uso no contexto brasileiro. Foi realizado um estudo observacional, transversal, em um hospital particular sem fins lucrativos, referência em segurança do paciente, localizado em uma metrópole brasileira. Os participantes foram selecionados a partir de uma amostra não-probabilística de todos os funcionários elegíveis de diversos departamentos hospitalares, convidados a participar no estudo. A confiabilidade do HSOPSC foi avaliada pela estimativa do alfa de Cronbach para cada dimensão. Foram usadas análise fatorial confirmatória (AFC), uma matriz de correlações entre as dimensões e modelagem de equações estruturais (MEE) na análise exploratória e confirmatória da validade estrutural do construto. Houve uma taxa global de resposta de 18,7% (n = 1.439). Quatro dimensões (“percepção geral de segurança do paciente”; “recursos humanos”; “trabalho de equipe entre unidades” e “resposta não-punitiva aos erros”) apresentaram problemas de consistência interna. A AFC mostrou um ajuste aceitável em relação ao modelo original de 12 dimensões. As correlações entre as dimensões do modelo original de 12 dimensões indicaram problemas de validade discriminante, enquanto a variância residual foi maior de 0,70 em 13 itens. A MEE do modelo original de 12 dimensões mostrou bom ajuste, com os seguintes índices: CFI = 0,985; TLI = 0,968 e RMSEA = 0,026 (IC90%: 0,024-0,029). Embora tenham superado a primeira avaliação, os resultados obtidos nesta reavaliação da validade e confiabilidade da versão do questionário traduzida e adaptada para o contexto brasileiro indicam a necessidade de mais estudos.

Palavras-chave:
Segurança do Paciente; Cultura Organizacional; Pesquisas sobre Serviços de Saúde; Reprodutibilidade dos Testes

Resumen:

El estudio tuvo como objetivo revisar las propiedades psicométricas de la versión del Hospital Survey on Patient Safety Culture (HSOPSC) adaptado al portugués, para su uso en el contexto brasileño. Se realizó un estudio transversal observacional, en un hospital grande con cuidados intensivos, privado, sin ánimo de lucro, punto de referencia en seguridad del paciente, situado en una de las mayores metrópolis brasileñas. Los participantes se seleccionaron de un muestreo no probabilístico entre todo el personal elegible de los diversos departamentos hospitalarios invitados a participar en el estudio. La fiabilidad del HSOPSC se evaluó estimando el alfa de Cronbach para cada dimensión. Se utilizaron tanto el análisis factorial confirmatorio (AFC), como la matriz de correlaciones entre las dimensiones y los modelos de ecuaciones estructurales exploratorios (ESEM) en el análisis exploratorio y confirmatorio para la validez estructural del constructo. El porcentaje general de respuesta fue de un 18,7% (n = 1,439). Hubo cuatro dimensiones (“percepciones generales de seguridad del paciente”; “dotación de personal”; “trabajo en equipo en las unidades”; y “respuesta al error no-punitiva”), problemas recurrentes de consistencia interna. El AFC mostró un ajuste aceptable con el modelo original de 12-dimensiones. Las correlaciones entre las dimensiones del modelo original de 12-dimensiones indicaron problemas de validez discriminante, mientras que varianza residual era mayor a 0,70 en 13 ítems. Los ESEM del modelo original de 12-dimensiones tuvieron un buen ajuste con los siguientes índices: CFI = 0,985; TLI = 0,968 y RMSEA = 0,026 (90%CI: 0,024-0,029). A pesar de que eran mejores que los de la primera evaluación, los resultados obtenidos en esta revisión de la validez y fiabilidad de la versión del cuestionario, traducido y adaptado al contexto brasileño, es necesario que haya más investigaciones al respecto sobre este asunto.

Palabras-clave:
Seguridad del Paciente; Cultura Organizacional; Encuestas de Atención de Salud; Reproducibilidad de los Resultados

Background

Safety culture comprises values, beliefs, and standards regarding what is important in an organization and what safety-related attitudes and behavior are valued, supported, and expected ¹. It is a multidimensional concept, defined in the health service context as the product of individual and group values, attitudes, perceptions, competences, and behavior patterns that determine the commitment to, and the style and proficiency of, an organization’s patient safety management ².

In health care organizations, safety culture assessment is a key strategy in the world movement to improve health care quality and patient safety. There is evidence of the association between high safety culture scores and positive clinical outcomes, especially in hospitals, as a result of interventions to improve the quality of patient care ^3,4,5.

Among the instruments developed to assess safety culture in hospitals, the Hospital Survey on Patient Safety Culture (HSOPSC) is one of the most widely used ^1,6. Developed in American English by the Agency for Healthcare Research and Quality (AHRQ), this instrument contains 42 items and 12 dimensions and was conceived as a questionnaire to be self-administered, to ensure respondents’ anonymity. The theoretical model underpinning its development is fully documented by the AHRQ ¹. Since its introduction in 2004, the HSOPSC has been used in 93 countries and some 40 translated versions exist ⁷. In psychometric analysis, the dimensional structure of the original model of the instrument has been confirmed by several studies ^{8,9,10,11,12,13} and underwent alterations in others ^14,15,16.

In Brazil, the transcultural adaptation of the HSOPSC was previously studied ^17,18 in a sample of 322 participants at two large (more than 149 beds), non-profit, public and private, acute-care general hospitals in two Brazilian states of the Southeast Region (one in Minas Gerais, with no history of quality improvement initiatives, and the other in the city of Rio de Janeiro, in preparation to submit to the hospital accreditation process). Confirmatory factor analysis (CFA) did not uphold the original 12-dimension, 42-item model of the instrument, but the exploratory nature of that study, the context in which the questionnaire was administered, and the sample may all have influenced that finding. Exploratory factor analysis returned a satisfactory 10-factor, 38-item model. However, the limitations of the study recommended caution when applying changes to the original model, pointing to a need to reassess the original 12-dimension, 42-item HSOPSC model adapted for use in the Brazilian context.

Considering the widespread use of the HSOPSC in diverse socioeconomic and cultural contexts and in comparative approaches, including organizations with more mature quality and patient safety cultures, this study aimed to reassess the reliability and validity of the original version of the HSOPSC adapted to Portuguese ¹⁷ for use in the Brazilian context and administered to the personnel of a hospital that is a reference in the area of patient safety.

Methods

Study design and setting

This quantitative, observational, cross-sectional study was conducted at a large, private, non-profit, acute-care hospital in a major metropolis in Brazil’s Southeast Region. Since the mid-1990s, this facility has invested in quality and subscribed to certification and accreditation programes, prioritizing quality of care and especially patient safety.

Sample

This was a non-probability sample, comprising all the eligible personnel present or on duty at the hospital during the data collection period. The personnel of various departments of the hospital were invited to participate, and data were collected electronically from September 1st to November 1st, 2013. The inclusion criteria followed the recommendations of the instrument’s developers ¹: participants should work at least 20 hours per week at the hospital.

Participation in the study was voluntary and, after agreeing to participate in the research and with their anonymity guaranteed, participants completed the questionnaire electronically. As a strategy to maximize the percentage response rate, communication took place via intranet. This communication channel was chosen to foster awareness among health care personnel, leaders, and managers in the various hospital departments of the importance of the subject and the focus and objectives of the study, to, subsequently, allow the instrument to be completed online.

Data collection instrument

The instrument used was the HSOPSC translated into Portuguese and adapted for use in Brazil ¹⁷. The HSOPSC contains 12 dimensions and 42 items and takes about 15 minutes to complete. Seven dimensions are related to features of the hospital department or unit where the respondent works: (i) communication openness (3 items); (ii) feedback and communication about error (3 items); (iii) organizational learning and continuous improvement (3 items); (iv) supervisor/manager expectations and actions promoting patient safety (4 items); (v) non-punitive response to errors (3 items); (vi) teamwork within units (4 items); and (vii) staffing (4 items). Three dimensions measure hospital-wide safety culture: (i) management support for patient safety (3 items); (ii) handoffs & transitions (4 items); and (iii) teamwork across units (4 items). There are two other output dimensions: (i) overall perceptions of patient safety (4 items); and (ii) frequency of events reported (3 items).

The HSOPSC also contains questions about the respondent’s characteristics (sex, occupational category, education level, and time of employment at the hospital). Most items are answered on a five-point Likert-type scale reflecting level of agreement, from “strongly disagree” (1) to “strongly agree” (5), with a neutral category “neither agree nor disagree” (3). Other items are answered on a five-point frequency scale from “never” (1) to “always” (5) ¹. Most of the questionnaire items are written positively, but 15 are worded negatively and reverse coded so that the highest scores indicate positive responses regarding patient safety ¹.

Data analysis

The construct was examined for reliability and structural validity. The analyses used only complete questionnaires and no strategy was applied to treat for missing data. Reliability is the degree to which the results obtained from the HSOPSC in the study sample could be replicated. To compare the results with previous psychometric assessments ^{8,10,12,13,18,19,20,21,22,23,24,25}, the reliability of the HSOPSC was assessed through its internal consistency using the Cronbach’s alpha estimated for each dimension, with values above 0.6 considered acceptable ²⁰.

In analyzing for structural validity, at the confirmatory and exploratory stages, CFA ²⁶, a correlation matrix between the dimensions, and exploratory structural equation modelling (ESEM) were used. The ESEM was chosen because of its ability to combine the characteristics of exploratory and confirmatory factor analysis, which makes it possible to explore the underlying factorial structure without the constraints imposed by CFA, in addition to offering CFA’s parameter and goodness-of-fit tests ²⁷.

First of all, the dimensional structure of the original 12-dimension instrument was assessed by CFA. Since the dependent variables in the HSOPSC are ordinal (five-point, Likert-type response options), the weighted least squares mean and variance-adjusted (WLSMV) estimator was used. In CFA, model fit was evaluated by three indices: root mean square error of approximation (RMSEA), comparative fit index (CFI), and Tucker-Lewis index (TLI). Absolute model fit was evaluated by RMSEA, in agreement with the use of the WLSMV estimator. RMSEA measures poor quality of fit by incorporating a penalty function to evaluate the lack of parsimony in the model’s degrees of freedom. The lower the RMSEA value, the better the fit: less than 0.05 is “good”, while values greater than 0.10 indicate “poor” fit and that the model should be rejected. The CFI and TLI indices, used as measures of incremental fit, enable the proposed model to be compared with an independent (null) model; the values of these indices range from 0 to 1, with values higher than 0.90 indicating an appropriate model ²⁶.

Subsequently, the dimensional structure was explored by ESEM, using the WLSMV estimator and oblique rotation (GEOMIN). Item factor loadings were also examined at this stage, with values from 0.35 to 0.50 considered acceptable; residual variance (uniqueness) above 0.70 was considered inadequate, as was the presence of cross-loading with a difference of more than 0.20 between loadings; in analyzing correlation between factors to determine discriminant validity, a value of 0.85 or higher was considered unsuitable ²⁸.

SPSS version 17.0 (https://www.ibm.com/br-pt/products/spss-statistics) was used to build the database, manage data, treat variables, perform descriptive statistical analyses and reliability calculations, and examine data factorability. Mplus version 7.0 (https://www.statmodel.com/) was used to calculate correlations between the dimensions, CFA, and ESEM.

This study was approved by the Ethics Research Committee of the Sergio Arouca National School of Public Health, Oswaldo Cruz Foundation (process n. 177/2011).

Results

Study sample and descriptive statistics

Of the 7,140 personnel invited to participate, 1,512 responded to the questionnaire. After exclusion of 28 questionnaires of participants who declared having no intention to collaborate and 45 records with more than 30% missing data, the final sample totalled 1,439 questionnaires (response rate = 18.7%).

The study sample comprised a young population, of which 82.7% (n = 1,082) were under 31 years old. Most respondents reported having completed higher education (99%; n = 1,422), being female (73.4%; n = 1,044), having direct contact with patients (89.8%; n = 1,320), and having worked at the hospital for more than three years (60.3%; n = 884).

The study participants were from various different hospital departments: intensive care unit (14.6%; n = 215); clinical surgery (10.7%; n = 157); internal medicine (7.6%; n = 112); emergency department (5.7%; n = 84); pharmacy (5.6%; n = 83); laboratory (5.6%; n = 82); rehabilitation (4.2%; n = 62); radiology (3.6%; n = 53); pediatrics (3.3%; n = 49); obstetrics (2.5%; n = 37); anesthesiology (0.5%; n = 8); psychiatry/mental health (0.3%; n = 4); and others.

The largest groups of participants, by occupational category, were nursing technicians (26.2%; n = 377) and nurses (20.3%; n = 292). Other categories of participant were: physical therapists/occupational therapists (9.1%; n = 131); administrative auxiliaries (6.5%; n = 93); pharmacists/biochemists/biologists (4.8%; n = 69); laboratory, pharmacy, x-ray, and electrocardiograph technicians (5.1%; n = 74); doctors (4.2%; n = 60); managers/director (3.2%; n = 46); nutritionists (1%; n = 14); psychologists (0.9%; n = 13); and others (18.6%; n = 268).

Internal consistency

The Cronbach’s alpha coefficients estimated for the original 12-factor model ranged from 0.41 (“overall perceptions of patient safety”) to 0.91 (“frequency of events reported”) (Table 1). Table 1 shows the results obtained in this study and in other international studies ^{8,10,12,13,18,19,20,21,22,23,24,25}. In four dimensions, the alpha value was below 0.60 (“overall perceptions of patient safety” = 0.41; “staffing” = 0.46; “teamwork across units” = 0.45; “non-punitive response to error” = 0.54) (Table 1).

Table 1
Internal consistency (Cronbach’s alpha) of the Hospital Survey on Patient Safety Culture (HSOPSC) dimensions.

Dimension/Items	Studies
	Current study	Brazil 18	Brazil e-version 19	USA 20	Netherlands 21	Norway 8	Switzerland 22	UK 23	Scotland 24	Japan 10	Slovenia 12	Portugal 13	France 25
Unit-level safety culture
Supervisor expectations and actions promoting patient safety (B1, B2, B3R, B4R)	0.74	0.76	0.73	0.75	0.70	0.77	0.79	0.68	0.79	0.70	0.74	0.72	0.75
Organizational learning - continuous improvement (A6, A9, A13)	0.62	0.56	0.61	0.68	0.57	0.51	0.68	0.66	0.64	0.65	0.36	0.71	0.57
Teamwork within units (A1, A3, A4, A11)	0.72	0.66	0.70	0.74	0.66	0.77	0.75	0.73	0.80	0.83	0.74	0.73	0.80
Communication openness (C2, C4, C6R)	0.69	0.69	0.59	0.64	0.72	0.68	0.64	0.67	0.73	0.62	0.74	0.67	0.67
Feedback/communication about error (C1, C3, C5)	0.68	0.72	0.74	0.81	0.75	0.70	0.78	0.80	0.78	0.77	0.72	0.76	0.77
Non-punitive response to error (A8R, A12R, A16R)	0.54	0.35	0.53	0.74	0.69	0.64	0.70	0.65	0.77	0.71	0.61	0.57	0.60
Staffing (A2, A5R, A7R, A14R)	0.46	0.20	0.59	0.70	0.49	0.65	0.65	0.58	0.60	0.46	0.65	0.48	0.61
Hospital-level safety culture
Hospital management support for patient safety (F1, F8, F9R)	0.69	0.84	0.74	0.84	0.68	0.79	0.83	0.69	0.79	0.61	0.82	0.77	0.79
Teamwork across units (F2R, F4, F6R, F10)	0.45	0.67	0.67	0.75	0.59	0.65	0.77	0.70	0.70	0.70	0.74	0.73	0.71
Handoffs and transitions (F3R, F5R, F7R, F11R)	0.72	0.70	0.73	0.75	0.68	0.65	0.72	0.77	0.74	0.73	0.66	0.71	0.78
Outcome measures
Overall perceptions of patient safety (A10R, A15, A17R, A18)	0.41	0.52	0.63	0.77	0.79	0.76	0.76	0.67	0.84	0.62	0.65	0.62	0.68
Frequency of event reported (D1, D2, D3)	0.91	0.91	0.87	0.88	0.62	0.82	0.70	0.83	0.71	0.88	0.88	0.90	0.86

• Note: item with R = reverse items.

Confirmatory factor analysis

CFA showed the fit of the original 12-dimension model to range from acceptable (CFI = 0.92 and TLI = 0.91) to satisfactory (RMSEA = 0.04) (90%CI: 0.042-0.046) (Table 2). That result is similar to those of international studies ^{10,12,13,18,19,20,21,22,23,24}, but differs from the findings of a previous Brazilian study ¹⁸ concerning the RMSEA value (Table 2).

Table 2
Confirmatory factor analysis of the 12 dimensions and 42 items: model fit index.

Index	Current study	Brazil 18	Brazil e-version 19	USA 20	Netherlands 21	Scotland 24	Switzerland 22	UK 23	Japan 10	Slovenia 12	Portugal 13
TLI	0.91	0.93	0.94	0.88	0.88	0.91	0.85	0.91	0.88	0.87	0.86
CFI	0.92	0.89	0.94	0.90	0.90	0.93	0.87	0.93	0.89	0.89	0.88
RMSEA	0.04	0.10	0.05	0.05	0.05	0.05	0.06	0.05	0.05	0.05	0.05
Sample	1,439	219	863	1,171	974	974	974	974	6,395	976	573

• CFI: comparative fit index; RMSEA: root mean square error of approximation; TLI: Tucker-Lewis.

Excessively high correlations (> 0.85) were observed between the dimensions “staffing” and “overall perceptions of patient safety” (0.89), between the dimensions “communication openness” and “feedback and communication about error” (0.94), and between the dimensions “handoffs and transitions” and “teamwork across units” (0.89) (Table 3). In five items (A5R, A7R, A10R, A16R and F10), loading was lower than 0.32. Residual variance was greater than 0.70 in 13 items (A5R, A7R, A8R, A10R, A11, A14R, A15, A16R, A17R, C1, F3R, F8 and F10), ranging from 0.71 to 1.0.

Regarding the high correlations observed, which occur often when true cross-loadings are constrained to be zero, such as in the CFA, a preliminary Bayesian second-order factor model was tested, assuming that a higher overall (safety culture) order might be influencing indirectly the manifest variables through the first order. The model was estimated with 10,000 total iterations, 5,000 burn-in and normal prior distribution for model parameters. Results showed that the model did not converge properly with convergence statistics = 1.0254, posterior predictive p-value = 0.0, and deviance information criteria = 8,739.53. Further Bayesian analysis must be performed in the future considering alternative hierarchical models.

Table 3
Correlation matrix estimated by confirmatory factor analysis for the latent variables of the original 12-dimension model.

	DIM 1	DIM 2	DIM 3	DIM 4	DIM 5	DIM 6	DIM 7	DIM 8	DIM 9	DIM 10	DIM 11
DIM 2	0.11
DIM 3	0.68	0.22
DIM 4	0.09	0.65	0.16
DIM 5	0.63	0.29	0.71	0.11
DIM 6	0.05	0.68	0.16	0.61	0.19
DIM 7	0.07	0.82	0.18	0.63	0.18	0.94
DIM 8	0.01	0.34	0.05	0.41	0.15	0.63	0.49
DIM 9	0.12	0.63	0.11	0.75	0.20	0.56	0.71	0.331
DIM 10	0.53	0.17	0.59	0.11	0.89	0.14	0.15	0.028	0.11
DIM 11	0.10	0.42	0.08	0.55	0.18	0.38	0.54	0.240	0.89	0.01
DIM 12	0.44	0.09	0.46	0.05	0.57	0.07	0.12	0.020	0.16	0.58	0.17

• DIM: dimension.
• DIM 1: teamwork within units; DIM 2: supervisor expectations and actions promoting patient safety; DIM 3: organizational learning - continuous improvement; DIM 4: hospital management support for patient safety; DIM 5: overall perceptions of patient safety; DIM 6: feedback and communication about error; DIM 7: communication openness; DIM 8: frequency of events reported; DIM 9: teamwork across units; DIM 10: staffing; DIM 11: handoffs and transitions; DIM 12: non-punitive response to error.

Exploratory structural equation modelling

ESEM returned good fit for the original 12-dimension model. The indices - CFI = 0.99; TLI = 0.97, and RMSEA = 0.026 (90%CI: 0.024-0.029) - were better than those returned by CFA.

The factorial structure indicated items with low loading (A8R “Staff feel like their mistakes are held against them”); cross loading (A16R “Staff worry that mistakes they make are kept in their personnel file”; A17R “We have patient safety problems in this unit”; A18 “Our procedures and systems are good at preventing errors from happening”; B4R “My supervisor/manager overlooks patient safety problems that happen repeatedly”; F2R “Hospital units do not coordinate well with each other”; F6R “It is often unpleasant to work with staff from other hospital units”; F8 “The actions of hospital management show that patient safety is a top priority”); and low and cross loading (F3R “Things ‘fall between the cracks’ when transferring patients from one unit to another”) (Table 4).

Table 4
Exploratory structural equation model, factorial loadings by component items of the dimensions aggregated according to the original model’s dimensional structure (42 items and 12 dimensions).

Dimension/Items	DIM 1	DIM 2	DIM 3	DIM 4	DIM 5	DIM 6	DIM 7	DIM 8	DIM 9	DIM 10	DIM 11	DIM 12
Teamwork within units
People support one another in this unit (A1)	0.849	-0.011	-0.003	-0.056	0.057	0.026	-0.004	0.011	0.008	-0.011	0.000	-0.011
When a lot of work needs to be done quickly, we work together as a team to get the work done (A3)	0.684	0.058	0.119	-0.014	-0.001	-0.03	0.006	0.006	0.013	-0.006	0.019	-0.05
In this unit, people treat each other with respect (A4)	0.673	-0.006	0.073	0.110	-0.034	-0.067	0.040	0.042	0.033	0.052	-0.034	0.035
When one area in this unit gets really busy, others help out (A11) *	0.378	0.260	-0.007	0.018	0.052	-0.005	-0.019	-0.008	-0.006	-0.027	-0.007	-0.001
Supervisor expectations and actions promoting patient safety
My supervisor/manager pays a compliment when he/she sees a job done according to established patient safety procedures (B1)	0.024	0.027	-0.004	0.015	-0.001	0.023	0.062	0.788	0.008	-0.037	0.023	0.029
My supervisor/manager gives serious consideration to staff suggestions for improving patient safety (B2)	0.040	-0.015	-0.003	-0.005	-0.009	0.147	-0.012	0.682	-0.007	0.036	0.138	0.008
Whenever pressure builds up, my supervisor/manager wants us to work faster, even if it means taking shortcuts (B3R)	-0.046	0.155	-0.138	-0.011	0.407	0.000	-0.056	0.096	0.051	0.175	-0.085	-0.025
My supervisor/manager overlooks patient safety problems that happen repeatedly (B4R)	-0.043	0.381	0.095	0.052	0.075	-0.106	0.065	0.109	0.050	0.219	-0.003	0.062
Organizational learning - continuous improvement
We are actively doing things to improve patient safety (A6)	0.241	0.145	0.482	0.061	0.024	0.069	-0.054	-0.017	-0.020	0.043	0.012	-0.032
A9 Mistakes have led to positive changes here	0.227	-0.019	0.493	0.065	0.160	0.035	-0.042	0.022	-0.065	-0.102	0.016	-0.012
After we make changes to improve patient safety, we evaluate their effectiveness (A13)	0.060	0.088	0.585	0.011	-0.068	0.061	0.018	-0.019	0.053	-0.038	-0.079	0.024
Hospital management support for patient safety
Hospital management provides a work climate that promotes patient safety (F1)	0.019	-0.012	-0.047	0.013	0.010	0.016	-0.109	0.003	-0.050	0.604	-0.397	-0.008
The actions of hospital management show that patient safety is a top priority (F8)	0.074	0.069	-0.035	0.041	-0.108	0.034	0.017	0.028	0.447	0.294	0.224	0.014
Hospital management seems interested in patient safety only after an adverse event happens (F9R)	0.007	0.015	-0.045	0.115	-0.084	0.035	-0.084	0.017	0.506	0.006	0.128	0.174
Overall perceptions of patient safety
It is just by chance that more serious mistakes do not happen here (A10R) *	-0.002	-0.019	0.014	0.506	-0.072	0.007	-0.043	0.069	-0.032	-0.038	0.064	-0.054
Patient safety is never sacrificed to get more work done (A15)	0.069	0.609	0.016	-0.131	0.035	-0.004	0.015	-0.051	-0.019	0.006	0.063	-0.015
We have patient safety problems in this unit (A17R)	-0.055	0.414	-0.102	0.316	0.072	0.102	-0.021	0.042	-0.012	-0.022	-0.103	-0.040
Our procedures and systems are good at preventing errors from happening (A18)	0.102	0.375	0.303	-0.014	-0.111	-0.087	0.047	0.074	0.015	0.010	-0.014	0.030
Feedback & communication about error
We are given feedback about changes put into place based on event reports (C1)	-0.035	0.019	-0.004	0.056	-0.035	0.512	0.087	0.026	0.027	0.150	-0.001	-0.067
We are informed about errors that happen in this unit (C3)	0.000	0.017	0.006	0.067	-0.027	0.643	0.148	-0.037	-0.061	-0.029	0.018	0.041
In this unit, we discuss ways to prevent errors from happening again (C5)	-0.002	0.014	-0.011	-0.054	-0.011	0.514	0.189	0.110	0.013	-0.009	0.161	-0.009
Communication openness
Staff will freely speak up if they see something that may negatively affect patient care (C2)	0.041	-0.067	0.056	0.026	-0.001	0.583	0.059	0.021	0.025	0.007	0.142	0.077
Staff feel free to question the decisions or actions of those with more authority (C4)	-0.018	0.005	0.021	-0.059	0.048	0.544	-0.019	0.277	0.104	0.17	-0.084	0.004
Staff are afraid to ask questions when something does not seem right (C6R)	-0.105	-0.009	0.125	-0.121	0.15	0.001	0.023	0.072	-0.017	0.133	0.443	0.413
Frequency of event reported
When a mistake is made, but is identified and corrected before affecting the patient, how often is this reported? (D1)	0.014	0.051	-0.022	-0.031	-0.028	0.047	0.826	0.006	0.005	-0.002	0.017	-0.015
When a mistake is made, but has no potential to harm the patient, how often is this reported? (D2)	−0.019	−0.015	0.024	−0.011	−0.002	0.042	0.875	−0.038	0.019	0.018	0.027	0.029
When a mistake is made that could harm the patient, but does not, how often is this reported? (D3)	0.003	-0.020	-0.018	0.043	0.032	-0.006	0.742	0.051	-0.015	-0.517	-0.016	-0.012
Teamwork across units
Hospital units do not coordinate well with each other (F2)	-0.030	0.018	0.012	0.075	-0.049	0.022	-0.022	0.152	-0.007	-0.070	0.378	0.469
There is good cooperation among hospital units that need to work together (F4)	0.043	0.004	-0.045	0.023	-0.015	0.159	-0.002	0.045	0.129	-0.108	0.541	-0.060
It is often unpleasant to work with staff from other hospital units (F6)	0.070	0.043	-0.070	-0.028	-0.026	0.01	-0.001	-0.068	0.397	0.016	-0.041	0.402
Hospital units work well together to provide the best care for patients (F10)	0.037	-0.023	-0.090	0.024	0.032	0.048	0.027	-0.058	0.022	0.001	0.610	0.000
Staffing
We have enough staff to handle the workload (A2)	0.182	0.506	0.082	0.009	-0.004	0.030	-0.024	-0.041	-0.052	0.004	0.057	0.031
Staff in this unit work longer hours than is best for patient care (A5R) *	0.005	0.072	-0.004	0.091	-0.155	0.040	-0.018	0.009	-0.024	-0.455	-0.036	0.042
We use more agency/temporary staff than is best for patient care (A7R) *,**	0.081	-0.003	-0.117	-0.026	0.206	0.130	0.004	0.037	-0.072	0.358	-0.158	0.014
We work in “crisis mode,” trying to do too much too quickly (A14R)	0.063	-0.155	0.126	0.541	0.059	-0.035	0.031	-0.032	-0.017	0.024	0.045	0.066
Handoffs & transitions
Things “fall between the cracks” when transferring patients from one unit to another (F3R) **	-0.047	0.001	0.083	-0.032	0.052	0.343	-0.044	0.027	0.221	-0.073	0.079	0.144
Important patient care information is often lost during shift changes (F5R)	0.045	-0.046	-0.074	0.035	0.021	0.013	0.034	-0.02	0.529	-0.038	-0.070	0.363
Problems often occur in the exchange of information across hospital units (F7R)	-0.040	-0.004	0.142	-0.020	0.039	-0.011	0.007	0.046	0.684	-0.069	0.042	-0.029
Shift changes are problematic for patients in this hospital (F11R)	-0.039	-0.046	0.143	-0.070	0.096	-0.001	0.023	-0.005	0.756	0.100	0.009	-0.049
Non-punitive response to error
Staff feel like their mistakes are held against them (A8R) *,**	-0.142	0.117	0.056	0.304	0.017	-0.018	0.041	-0.022	0.053	0.276	-0.019	-0.060
When an event is reported, it feels like the person is being written up, not the problem (A12R)	0.082	0.033	0.003	0.069	0.606	-0.014	0.034	-0.016	0.021	0.053	0.037	-0.013
Staff worry that mistakes they make are kept in their personnel file (A16R)	0.094	0.097	0.047	0.338	0.442	0.007	0.003	-0.051	0.017	-0.111	0.002	0.071

• Note: item with R = reverse items.
• * Residual variance > 0.70;
• ** Loading below 0.35.

Of the 12 dimensions, four - “teamwork within units,” “organizational learning,” “feedback and communication about error”, and “frequency of events reported” (outcome dimension) - loaded on all the items satisfactorily in the dimensional structure of the original HSOPSC model. The reverse items - A14R (“We work in ‘crisis mode’ trying to do too much too quickly”); A10R (“It is just by chance that more serious mistakes do not happen here”); and A8R (“Staff feel like their mistakes are held against them”) - were observed to load on the same factor, thus constituting a dimension, and two of whose items are related to the hospital department/unit and the other, to the outcome dimension (Table 4).

Table 5 shows the dimensional structure resulting from the ESEM for the study sample. Compared to the dimensional structure of the original model, the items of the dimensions “feedback and communication about error” and “communication openness” intersect with each other, as do those of “management support for patient safety” and “handoffs & transition”. Items of the dimensions “supervisor/manager expectations & actions promoting patient safety”, “management support for patient safety”, “communication openness”, “staffing”, and “non-punitive response to errors” loaded on other dimensions, producing dimensions comprising only two items, which is an undesirable configuration. On this criterion, the dimension “teamwork across units” displayed the most problematic factorial structure, with four items loading on three different dimensions.

Residual variance, although mostly borderline, was high in the following items: A11 (“When one area in this unit gets really busy, others help out”) (0.71) of the dimension “teamwork within units”; A8R (“Staff feel like their mistakes are held against them”) (0.83) of the dimension “non-punitive response to error”; A10R (“It is just by chance that more serious mistakes do not happen here”) (0.71) of the dimension “teamwork across units”; A7R (“We use more agency/temporary staff than is best for patient care”) (0.71) of the dimension “staffing”; and A5R (“Staff in this unit work longer hours than is best for patient care”) (0.71), also of the dimension “staffing”.

ESEM with 12 dimensions returned correlations that were not excessively high, indicating no problems of discriminant validity.

Table 5
Dimensional structure from the exploratory structural equation model, factorial loadings by component items.

Item	DIM 1	DIM 2	DIM 3	DIM 4	DIM 5	DIM 6	DIM 7	DIM 8	DIM 9	DIM 10	DIM 11	DIM 12
People support one another in this unit (A1)	0.849	-0.011	-0.003	-0.056	0.057	0.026	-0.004	0.011	0.008	-0.011	0.000	-0.011
When a lot of work needs to be done quickly, we work together as a team to get the work done (A3)	0.684	0.058	0.119	-0.014	-0.001	-0.030	0.006	0.006	0.013	-0.006	0.019	-0.05
In this unit, people treat each other with respect (A4)	0.673	-0.006	0.073	0.110	-0.034	-0.067	0.040	0.042	0.033	0.052	-0.034	0.035
When one area in this unit gets really busy, others help out (A11)	0.378	0.260	-0.007	0.018	0.052	-0.005	-0.019	-0.008	-0.006	-0.027	-0.007	-0.001
Patient safety is never sacrificed to get more work done (A15)	0.069	0.609	0.016	-0.131	0.035	-0.004	0.015	-0.051	-0.019	0.006	0.063	-0.015
We have patient safety problems in this unit (A17R)	-0.055	0.414	-0.102	0.316	0.072	0.102	-0.021	0.042	-0.012	-0.022	-0.103	-0.040
Our procedures and systems are good at preventing errors from happening (A18)	0.102	0.375	0.303	-0.014	-0.111	-0.087	0.047	0.074	0.015	0.010	-0.014	0.030
My supervisor/manager overlooks patient safety problems that happen repeatedly (B4R) *	-0.043	0.381	0.095	0.052	0.075	-0.106	0.065	0.109	0.050	0.219	-0.003	0.062
We have enough staff to handle the workload (A2)	0.182	0.506	0.082	0.009	-0.004	0.030	-0.024	-0.041	-0.052	0.004	0.057	0.031
We are actively doing things to improve patient safety (A6)	0.241	0.145	0.482	0.061	0.024	0.069	-0.054	-0.017	-0.020	0.043	0.012	-0.032
Mistakes have led to positive changes here (A9)	0.227	-0.019	0.493	0.065	0.160	0.035	-0.042	0.022	-0.065	-0.102	0.016	-0.012
After we make changes to improve patient safety, we evaluate their effectiveness (A13)	0.060	0.088	0.585	0.011	-0.068	0.061	0.018	-0.019	0.053	-0.038	-0.079	0.024
It is just by chance that more serious mistakes do not happen here (A10R)	-0.002	-0.019	0.014	0.506	-0.072	0.007	-0.043	0.069	-−0.032	-0.038	0.064	-0.054
We work in “crisis mode”, trying to do too much too quickly (A14R)	0.063	-0.155	0.126	0.541	0.059	-0.035	0.031	-0.032	-0.017	0.024	0.045	0.066
Staff feel like their mistakes are held against them (A8R) *	-0.142	0.117	0.056	0.304	0.017	-−0.018	0.041	-0.022	0.053	0.276	-0.019	-0.060
Whenever pressure builds up, my supervisor/manager wants us to work faster, even if it means taking shortcuts (B3R)	-0.046	0.155	-0.138	-0.011	0.407	0.000	-0.056	0.096	0.051	0.175	-0.085	-0.025
When an event is reported, it feels like the person is being written up, not the problem (A12R)	0.082	0.033	0.003	0.069	0.606	-0.014	0.034	-0.016	0.021	0.053	0.037	-0.013
Staff worry that mistakes they make are kept in their personnel file (A16R)	0.094	0.097	0.047	0.338	0.442	0.007	0.003	-0.051	0.017	-0.111	0.002	0.071
We are given feedback about changes put into place based on event reports (C1)	-0.035	0.019	-0.004	0.056	-0.035	0.512	0.087	0.026	0.027	0.150	-0.001	-0.067
We are informed about errors that happen in this unit (C3)	0.000	0.017	0.006	0.067	-0.027	0.643	0.148	-0.037	-0.061	-0.029	0.018	0.041
In this unit, we discuss ways to prevent errors from happening again (C5)	-0.002	0.014	-0.011	-0.054	-0.011	0.514	0.189	0.110	0.013	-0.009	0.161	-0.009
Staff will freely speak up if they see something that may negatively affect patient care (C2)	0.041	-0.067	0.056	0.026	-0.001	0.583	0.059	0.021	0.025	0.007	0.142	0.077
Staff feel free to question the decisions or actions of those with more authority (C4)	-0.018	0.005	0.021	-0.059	0.048	0.544	-0.019	0.277	0.104	0.170	-0.084	0.004
Things “fall between the cracks” when transferring patients from one unit to another (F3R) *	-0.047	0.001	0.083	-0.032	0.052	0.343	-0.044	0.027	0.221	-0.073	0.079	0.144
When a mistake is made, but is identified and corrected before affecting the patient, how often is this reported? (D1)	0.014	0.051	-0.022	-0.031	-0.028	0.047	0.826	0.006	0.005	-0.002	0.017	-0.015
When a mistake is made, but has no potential to harm the patient, how often is this reported? (D2)	-0.019	-0.015	0.024	-0.011	-0.002	0.042	0.875	-0.038	0.019	0.018	0.027	0.029
When a mistake is made that could harm the patient, but does not, how often is this reported? (D3)	0.003	-0.020	-0.018	0.043	0.032	-0.006	0.742	0.051	-0.015	-0.517	-0.016	-0.012
My supervisor/manager pays a compliment when he/she sees a job done according to established patient safety procedures (B1)	0.024	0.027	-0.004	0.015	-0.001	0.023	0.062	0.788	0.008	-0.037	0.023	0.029
My supervisor/manager gives serious consideration to staff suggestions for improving patient safety (B2)	0.040	-0.015	-0.003	-0.005	-0.009	0.147	-0.012	0.682	-0.007	0.036	0.138	0.008
The actions of hospital management show that patient safety is a top priority (F8)	0.074	0.069	-0.035	0.041	-0.108	0.034	0.017	0.028	0.447	0.294	0.224	0.014
Hospital management seems interested in patient safety only after an adverse event happens (F9R)	0.007	0.015	−0.045	0.115	-0.084	0.035	-0.084	0.017	0.506	0.006	0.128	0.174
Important patient care information is often lost during shift changes (F5R)	0.045	-0.046	-0.074	0.035	0.021	0.013	0.034	-0.020	0.529	-0.038	-0.070	0.363
Problems often occur in the exchange of information across hospital units (F7R)	-0.040	-0.004	0.142	-0.020	0.039	-0.011	0.007	0.046	0.684	-0.069	0.042	-0.029
Shift changes are problematic for patients in this hospital (F11R)	-0.039	-0.046	0.143	-0.070	0.096	-0.001	0.023	-0.005	0.756	0.100	0.009	-0.049

• Note: item with R = reverse items; grey shading highlights the higher loading of each item between dimensions
• * Loading below 0.35.

Discussion

The HSOPSC is widely used in countries with a diversity of socioeconomic and historical contexts regarding quality and patient safety. Considering the limitations of the previous study, which evaluated this instrument for reliability and validity in the Brazilian context ¹⁸, its properties were reexamined here. The study sample considered here comprised 1,439 questionnaires (an 18.7% response rate), from which data was collected electronically, via intranet, at a hospital with a history and leading role in initiatives in the field of quality and patient safety. The study sample was much larger than in the previous study, which collected data using questionnaires on paper, but the response rate was lower, with the previous study obtaining a response rate of 38.5%. Response rates are important, because low rates can limit the generalization of the data on safety culture to the level of the hospital where the instrument is used. Comparative data from hospital surveys have shown higher response rates when the instrument is applied on paper ¹. Andrade et al. ¹⁹ adapted and validated an electronic Brazilian version of the HSOPSC and recommended addressing the low response rate from in-person administration by enabling completion on mobile electronic applications. However, the AHRQ suggests that participants may be concerned about the privacy of their responses and may not be receptive to completing the questionnaire at their workplace ¹.

Nursing personnel accounted for the largest percentage of study participants by profession, corroborating the findings of a recent systematic review of studies that have used the HSOPSC ⁶. Awareness-building and encouragement strategies need to be directed to all occupational categories. Special attention is required, therefore, when assessing an organization’s overall safety culture.

Internal consistency

Four dimensions displayed problems of internal consistency (“overall perceptions of patient safety”; “staffing”; “teamwork across units”; “non-punitive response to error”). With the exception of the dimension “teamwork across units”, all dimensions had already returned inadequate results in the previous Brazilian study ¹⁸. In the validation of the electronic version for Brazil ¹⁹, three dimensions (“non-punitive response to error”, “staffing” and “communication openness”) returned values below 0.60. Unlike the international studies ^{8,10,12,20,21,22,23,24,25}, the Brazilian studies ^18,19, the Portuguese study ¹³, and this study highlighted the dimension “non-punitive response to error” as showing internal consistency problems. The dimensions “overall perceptions of patient safety”, “staffing”, “teamwork across units”, and “non-punitive response to error” contain reverse items. Even though the strategy of interspersing reverse-phrased sentences is effective in reducing tendentious responses, the AHRQ states that HSOPSC users have reported difficulty in interpreting them and the agency plans to reduce the number of reverse items when the instrument is updated ²⁹.

Confirmatory factorial analysis

The CFA results in this study were both better than those of the previous study ¹⁸ and, unlike that study, similar to those of the international studies ^12,20,24 in confirming a 12-factor model. Note that the study context may explain that finding. However, there were problems of relevance and reliability in certain items (residual variance was higher than 0.70 in 13 items) and of discriminant validity (with excessively high correlations between the dimensions “staffing” and “overall perceptions of patient safety”, between the dimensions “communication openness” and “feedback and communication about error”, and between the dimensions “handoffs and transitions” and “teamwork across units”). Five items (A5R, A7R, A10R, F10 and A16R) returned loading below 0.32.

Exploratory structural equation modelling

ESEM found very good model fit, with better indices than those returned by CFA. However, high residual variance in the behaviour of at least five items indicated irrelevance. The factorial structure showed weakness, suggesting a 12-dimension model, two of whose dimensions would comprise only two items each (which is undesirable), with problematical loading on eight items (A8R, A16R, A18, B4R, F2R, F6R, F8, and F3R), six of which are reverse items, which - as mentioned above - have led to problems of comprehension among users of the instrument in other contexts.

Note that, with the exception of this study and the previous validation study by Reis et al. ¹⁸, studies of versions of the HSOPSC adapted for use in Brazil did not present data on correlations between the dimensions.

The version translated and adapted for use in the Brazilian context ¹⁷ did not attain the best results in all validity and reliability tests with the data obtained from this study sample, despite the fact that the universe of this study differed from the previous one in that the hospital in question has a proactive history in initiatives in the field of quality of care and patient safety. This leads the authors to question whether these findings resulted from the need to improve the translation and retest it or whether the psychometric problems are inherent to the instrument itself and not specific to the translation and adaptation of the version used in this study, as already inferred by other users of the HSOPSC ^25,29. Despite the latter question, certain points regarding the original version of the HSOPSC are worth stressing: (i) the instrument is applicable to clinical and non-clinical personnel in the hospital context, but does not offer the response option “not applicable/I don’t know”, which may create inappropriate responses on certain items, leading to inconsistencies in the assessment of the hospital’s culture; (ii) the HSOPSC focuses on non-punitive culture, which is probably consistent with the time at which the instrument was formulated, but there is now an emerging need to shift focus towards a culture of fairness that values learning rather than highlighting culpability, and thus favours understanding individual and system-related factors that contribute to the occurrence of safety incidents; and (iii) the HSOPSC is a very extensive instrument, containing many reverse items, which may make its questions not very clear to respondents.

However, subsequent analyses based on the ESEM results recommended excluding some nine items, which would result ultimately in dimensions comprising fewer than three items, which is inadvisable. Accordingly, considering the findings of the studies conducted in the Brazilian context, there is a need to examine the dimensional structure and the relevance of items.

The results reported here should be interpreted in light of the respondents, selected from a non-random sampling of a single hospital, with low response rates. Also, we were not able to examine if a possible selection bias might have occurred, since respondents were selected on a voluntary basis and we did not obtain data from non-respondents. These aspects may limit a generalization of our findings to other general hospitals in Brazil. Despite the limitations, this study aimed to test the instrument in a specific hospital context whose results could serve as reference for other studies that seek to implement safety strategies.

However, the larger purpose of measuring patient safety culture is to identify and strengthen measures to reduce unnecessary risks resulting from patient care. In that regard, the endeavour to establish an adaptation of the HSOPSC properly suited to the Brazilian context is important, because that will not only permit national and international comparisons, but can assist in monitoring recently introduced measures directed to fostering patient safety, which certainly may be more at risk in this conjuncture of financial crisis.

Conclusion

Although the results obtained from reassessing the validity and reliability of the version of the HSOPSC translated and adapted for use in the Brazilian context were better than those of the first assessment, they do need further investigation. Whether this is due to the translation process, the limits of this study, Brazilian particular characteristics and cultural diversity, or related to more general problems in the structure of the instrument cannot be ascertained here.

Acknowledgments

The study was supported by the Brazilian National Research Council (Edital Universal 14/2011, process n. 471764/2001-6; Bolsa PQ, process n. 306023/2016-4).

References

• ¹
  Sorra J, Gray L, Streagle S, Famolaro T, Yount N, Behm J. AHRQ Hospital Survey on Patient Safety Culture: user's guide. Rockville: Agency for Healthcare Research and Quality; 2016. (AHRQ Publication, 15-0049-EF).
• ²
  Nieva VF, Sorra J. Safety culture assessment: a tool for improving patient safety in healthcare organizations. Qual Saf Health Care 2003; 12 Suppl 2:ii17-23.
• ³
  Mardon RE, Khanna K, Sorra J, Dyer N, Famolaro T. Exploring relationships between hospital patient safety culture and adverse events. J Patient Saf 2010; 6:226-32.
• ⁴
  Di Cuccio MH. The relationship between patient safety culture and patient outcomes: a systematic review. J Patient Saf 2014; 22:11-8.
• ⁵
  Fan CJ, Pawlik TM, Daniels T, Vernon N, Banks K, Westby P, et al. Association of safety culture with surgical site infection outcomes. J Am Coll Surg 2016; 222:122-8.
• ⁶
  Reis CT, Paiva SG, Sousa P. The patient safety culture: a systematic review by characteristics of Hospital Survey on Patient Safety Culture dimensions. Int J Qual Health Care 2018; 30:660-77.
• ⁷
  Agency for Healthcare Research and Quality. International use of SOPS. http://www.ahrq.gov/professionals/quality-patient-safety/patientsafetyculture/pscintusers.html (accessed on 24/Nov/2017).
  » http://www.ahrq.gov/professionals/quality-patient-safety/patientsafetyculture/pscintusers.html
• ⁸
  Olsen E. Reliability and validity of the Hospital Survey on Patient Safety Culture at a Norwegian hospital. In: Øvretveit J, Sousa PJ, editors. Quality and safety improvement research: methods and research practice from the International Quality Improvement Research Network (QIRN). Lisboa: Escola Nacional de Saúde Pública, Universidade Nova de Lisboa; 2008. p. 173-86.
• ⁹
  Chen IC, Li HH. Measuring patient safety culture in Taiwan using the Hospital Survey on Patient Safety Culture (HSOPSC). BMC Health Serv Res 2010; 10:152.
• ¹⁰
  Ito S, Seto K, Kigawa M, Fujita S, Hasegawa T, Hasegawa T. Development and applicability of Hospital Survey on Patient Safety Culture (HSOPS) in Japan. BMC Health Serv Res 2011; 11:28.
• ¹¹
  Moghri J, Arab M, Saari AA, Nateqi E, Forooshani AR, Ghiasvand H, et al. The psychometric properties of the Farsi version of 'Hospital Survey on Patient Safety Culture' in Iran's hospitals. Iran J Public Health 2012; 42:80-6.
• ¹²
  Robida A. Hospital Survey on Patiet Safety Culture in Slovenia: a psychometric evaluation. Int J Qual Health Care 2013; 25:469-75.
• ¹³
  Eiras M, Escoval A, Grillo IM, Silva-Fortes C. The hospital survey on patient safety culture in Portuguese hospitals: instrument validity and reliability. Int J Health Care Qual Assur 2014; 27:111-22.
• ¹⁴
  Smits M, Wagner C, Spreeuwenberg P, van der Wal G, Groenewegen PP. Measuring patient safety culture: an assessment of the clustering of responses at unit level and hospital level. Qual Saf Health Care 2009; 18:292-6.
• ¹⁵
  Blegen MA, Gearhart S, O'Brien R, Sehgal NL, Alldredge BK. AHRQ's Hospital Survey on Patient Safety Culture: psychometric analyses. J Patient Saf 2009; 5:139-44.
• ¹⁶
  Bodur S, Filiz E. Validity and reliability of Turkish version of 'Hospital Survey on Patient Safety Culture' and perception of patient safety in publichospitals in Turkey. BMC Health Serv Res 2010; 10:28.
• ¹⁷
  Reis CT, Laguardia J, Martins M. Adaptação transcultural da versão brasileira do Hospital Survey on Patient Safety Culture: etapa inicial. Cad Saúde Pública 2012; 28:2199-210.
• ¹⁸
  Reis CT, Laguardia J, Vasconcelos AGG, Martins M. Reliability and validity of the Brazilian version of the Hospital Survey on Patient Safety Culture (HSOPSC): a pilot study. Cad Saúde Pública 2016; 32:e00115614.
• ¹⁹
  Andrade LEL, Melo LOM, Silva IG, Souza RM, Lima ALB, Freitas MR, et al. Adaptação e validação do Hospital Survey on Patient Safety Culture em versão brasileira eletrônica. Epidemiol Serv Saúde 2017; 26:455-68.
• ²⁰
  Sorra J, Nieva V. Hospital Survey on Patient Safety Culture. Rockville: Agency for Healthcare Research and Quality; 2004. (AHRQ Publication, 04-0041).
• ²¹
  Smits M, Christiaans-Dingelhoff I, Wagner C, Wal Gv, Groenewegen PP. The psychometric properties of the 'Hospital Survey on Patient Safety Culture' in Dutch hospitals. BMC Health Serv Res 2008; 8:230.
• ²²
  Pfeiffer Y, Manser T. Development of the German version of the Hospital Survey on Patient Safety Culture: dimensionality and psychometric properties. Saf Sci 2010; 48:1452-62.
• ²³
  Waterson P, Griffiths P, Stride C, Murphy J, Hignett S. Psychometric properties of the Hospital Survey on Patient Safety Culture: findings from the UK. Qual Saf Health Care 2010; 19:2e.
• ²⁴
  Sarac C, Flin R, Mearns K, Jackson J. Hospital survey on patient safety culture: psychometrics analysis on a Scottish sample. BMJ Qual Saf 2011; 20:842-8.
• ²⁵
  Perneger TV, Staines A, Kundig F. Internal consistency, factor structure and construct validity of the French version the Hospital Survey on Patient Safety Culture. BMJ Qual Saf 2014; 23:389-97.
• ²⁶
  Brown TA. Confirmatory factor analysis for applied research. New York: The Guilford Press; 2006.
• ²⁷
  Kelloay EK. Using Mplus for structural equation modeling: a researcher’s guide. 2nd Ed. Thousand Oaks: Sage Publications; 2015.
• ²⁸
  Reichenheim ME, Hokerberg YHM, Moraes CL. Assessing construct structural validity of epidemiological measurement tools: a seven-step roadmap. Cad Saúde Pública 2014; 30:927-39.
• ²⁹
  Agency for Healthcare Research and Quality. Update on version 2.0 of the Hospital SOPS. https://www.ahrq.gov/sops/quality-patient-safety/patientsafetyculture/hospital/update/index.html (accessed on 16/Aug/2018).
  » https://www.ahrq.gov/sops/quality-patient-safety/patientsafetyculture/hospital/update/index.html

Publication Dates

• Publication in this collection
  22 Aug 2019
• Date of issue
  2019

History

• Received
  19 Dec 2018
• Reviewed
  16 Apr 2019
• Accepted
  25 Apr 2019