AGRASS Questionnaire: Assessment of Risk Management in Health Care

Gama, Zenewton André da Silva; Saturno-Hernandez, Pedro Jesus; Caldas, Anna Claudia Sales Gomes; Freitas, Marise Reis de; Mendonça, Ana Elza Oliveira de; Medeiros, Carlos Alexandre de Souza; Medeiros, Wilton Rodrigues; Kessler, Oliver; Soares, Diogo Penha

doi:10.11606/s1518-8787.2020054001335

Acessibilidade / Reportar erro

Brasil

Español English

sumário « anterior atual seguinte »

Sumário

Original Article • Rev. Saúde Pública 54 • 2020 • https://doi.org/10.11606/s1518-8787.2020054001335 copy

AGRASS Questionnaire: Assessment of Risk Management in Health Care

Authorship SCIMAGO INSTITUTIONS RANKINGS

ABSTRACT

OBJECTIVE

This study aims to assess the development and the validity analysis of the Assessment of Risk Management in Health Care Questionnaire (AGRASS).

METHODS

This is a validation study of a measurement instrument following the stages: 1) Development of conceptual model and items; 2) Formal multidisciplinary assessment; 3) Nominal group for validity analysis with national specialists; 4) Development of software and national pilot study in 62 Brazilian hospitals 5) Delphi for validity analysis with the users of the questionnaire. In stages 3 and 5, the items were judged based on face validity, content validity, and utility and viability, by a 1-7 Likert scale (cut-off point: median < 6). Accuracy and reliability of the questionnaire were analyzed with the Confirmatory Factor Analysis and the Cronbach’s alpha.

RESULTS

The initial version of the instrument (98 items) was adapted during stages 1 to 3 for the final version with 40 items, which were considered relevant, of adequate content, useful, and viable. The instrument has 2 dimensions and 9 subdimensions, and the items have closed-ended questions (yes or no). The software for the automatic collection and analysis generates indicators, tables, and automatic graphs for the assessed institution and aggregated data. The adjustment indices confirmed a bi-dimensional model composed of structure and process (X2/gl = 1.070, RMSEA ≤ 0.05 = 0.847, TLI = 0.972), with high reliability for the AGRASS Questionnaire (α = 0.94) and process dimension (α = 0.93), and adequate for the structural dimension (α = 0.70).

CONCLUSIONS

The AGRASS Questionnaire is a potentially useful instrument for the surveillance and monitoring of the risk management and patient safety in health services.

Patient Safety; Risk Management; Validation Studies

Variable	Stage 3 ANVISA n=10		Stage 5 State and municipal visas N=32

	Number	Percentage	Number	Percentage
Sex
Female	6	60%	30	93.7%
Male	4	40%	2	6.3%
Age
< 30 years old	-	-	1	3.1%
31 to 40 years old	9	90%	11	34.4%
41 to 50 years old	1	10%	12	37.5%
51 to 60 years old	-	-	8	25.0%
Professional Training
Architecture	-	-	1	3.1%
Biology	1	10%	-	-
Nursing	4	40%	22	68.8%
Engineering	-	-	1	3.1%
Pharmacy	1	10%	3	9.4%
Physics	1	10%	-	-
Physical therapy	-	-	2	6.2%
Medicine	-	-	2	6.2%
Dentistry	3	30%	-	-
Collective Health	-	-	1	3.1%
Graduate Degree
None	1	10%	3	9.4%
Graduate Specialization	5	50%	19	59.4%
Masters	3	30%	8	25.0%
PhD	1	10%	2	6.2%
Service time
< 5 years	3	30%	11	34.4%
6 to 10 years	3	30%	9	28.1%
11 to 15 years	4	40%	7	21.8%
> 15 years	-	-	5	15.7%
NSSS scope*
National	10	100%	-	-
State	-	-	18	56.2%
Municipal	-	-	16	43.7%

VERSION 2	Median 1st Vote / 2nd Vote† n = 10				VERSION 3:	Median Single Vote† n = 32

	C1	C2	C3	C4		C1	C2	C3	C4
PART I – STRUCTURE FOR THE RISK MANAGEMENT ASSISTANCE					PART I – STRUCTURE FOR THE RISK MANAGEMENT ASSISTANCE
Awareness					Awareness
1. Did the hospital promote any awareness-raising action for patient safety in the last year (event, campaign, etc.)?	7	7	7	6	1. Has the institution promoted any awareness-raising action for patient safety in the last 12 months (event, campaign, etc.)?	7	7	7	7
2. Are there posters, folders, posters or videos in the hospital drawing attention to patient safety?	5/7	7/7	7/7	5/6	2. Are there in the institution posters, folders, posters or videos drawing attention to patient safety?	7	7	7	7
Accountability					Accountability
3. Is there an organizational unit responsible for improving patient safety (called from now on National Patient Safety Program – PNSP)?	7	7	7	7	3. Is there an organizational unit responsible for coordinating the actions of patient safety (called from now on to National Patient Safety Program – PNSP)?	7	7	7	7
4. Does it have a PNSP coordinator been appointed?	6	7	7	6,5	4. No adjustment	7	7	7	7
5. Does the PNSP have a record of at least six meetings in the last 12 months?	5/6	6/6	5/6	5/6	Eliminated after group discussion
6. Is there a National Patient Safety Plan running?	7	6,5	7	7	5. No adjustment	7	7	7	7
Ability					Ability
7. Does the NSP have professionals with formal workload dedication to risk management activities?	6,5	7	6,5	6	6. Does the institution have professionals with formal workload dedication to risk management activities?	7	7	7	7
8. Does the PNSP coordinator have exclusive dedication to risk management activities?	6/6	6/7	5/6	5/6	Eliminated after group discussion
9. Is there allocation of financial resource for promotion of patient safety actions?	6/6	6/6	4/5	6/5	Eliminated
10. Does the PNSP have adequate physical infrastructure?	5/6	5/6	4/6	4/7	7. No adjustment	7	7	7	7
11. Does the hospital provide adequate inputs for risk management actions?	4/6	6/6	5/7	5/7	8. Does the institution provide adequate inputs for risk management actions?	7	7	7	7
12. Did the hospital promote training to its professionals in the area (risk management, quality management, patient safety, etc.)?	7	7	7	7	9. Did the institution promote training to its professionals in the area (risk management, quality management, patient safety, etc.)?	7	7	7	7
Safety Culture					Promoting patient safety culture
13. Was the safety culture assessed in the last 12 months?	5/7	7/7	5/7	6/7	10. Was the patient safety culture assessed in the last 24 months ?	7	7	7	7
14. If it assessed the safety culture, were the results reported to clinical, administrative and care professionals?	5/7	7/7	7/7	6/7	11. Did it communicate the results of the patient’s safety culture assessment to clinical, administrative and care professionals?	7	7	7	7
15. If it assessed the safety culture, were implemented any interventions to improve the results identified in the assessment?	7	7	6,5	7	12. Did it implement any intervention to improve the results identified in the assessment of the patient’s safety culture?	7	7	7	7
PART 2 – PROCESSES FOR THE RISK MANAGEMENT ASSISTANCE Risk identification					PART 2 – PROCESSES FOR THE RISK MANAGEMENT ASSISTANCE Risk identification
16. Does the hospital have a general list of the care risks identified in the institution?	6	7	7	7	Eliminated after group discussion
17. Does it use an internal system to notify incidents?	6	7	6	7	13. No adjustment	7	7	7	7
18. Did the hospital disclose a list of sentinel events or never events for notification among professionals?	6	6,5	7	6	14. No adjustment	7	7	7	7
19. Does it monitor adherence to international patient safety goals?	6,5	6	6	6	15. Does it monitor indicators of adherence to international patient safety goals?	7	7	7	7
20. Does the hospital use information on complaints and appeals (ombudsman’s office) to identify risks?	7	7	7	7	16. No adjustment	7	7	7	7
21. Does it use triggers or result tracker indicators to identify risks?	7/6	7/7	5/7	6/7	17. Does it use tracker indicators ( clues to the existence of security incidents ) or result indicators to identify risks?	7	7	7	7
22. Does it have a death review committee with meetings in the last six months (minutes)?	6	6	6	6	18. Does it use the information on the death review committee to identify risks?	7	7	7	7
23. Does it use the litigious processes of the hospital for risk identification?	6/7	6/7	6/7	5/7	19. Does it analyzes the litigious processes of the hospital for risk identification?	7	7	7	7
24. Does it use direct observation to identify risks? (e.g. examination of hand hygiene, contact precautions with patients in isolation, protective barriers in the insertion of Central Venous Catheter, etc.)?	7	6	7	7	20. No adjustment	7	7	7	7
25. Does it use electronic alert or support system for decision-making in electronic medical records (e.g. drug interactions, standardized discharge recommendations for specific patients, etc.)?	5/6	5/7	6/6	7/6	21. Does it use electronic alert system in electronic medical records (e.g. drug interactions, standardized discharge recommendations for specific patients etc.)?	7	7	7	7
26. Does it use checklists for patient safety?	7	7	7	7	22. No adjustment	7	7	7	7
27. Does it use risk mapping?	7	7	7	7	23. Did it carry out risk mapping of the health service?	7	7	7	7
28. Do security officials conduct security rounds in sectors to identify risks?	6	7	7	6	24. Are patient safety rounds performed in the sectors to identify risks?	7	7	7	7
29. Does it confirm the use of external source to identify possible risks of at the institution (e.g. health alerts, media news, etc.)?	6	7	7	6	25. Does it use external source for risk identification (e.g. health alerts, media news, etc.)?	7	7	7	7
Risk assessment analysis					Risk assessment analysis
30. Did it investigate (analysis of causes and contributing factors) any adverse events in the last 12 months?	7	7	7	7	26. Did it perform analysis of causes and contributing factors for adverse events in the last 12 months?	7	7	7	7
31. Does it use instruments for qualitative analysis of causes and contributing factors (flowchart, cause-effect diagram, force-field analysis, Bow Tie, brainstorming, etc.)?	7	7	7	7	27. No adjustment	7	7	7	7
32. Does it use instruments for quantitative analysis of causes or contributing risk factors (histogram, stratification, Pareto diagram and control chart)?	7	7	7	7	28. No adjustment	7	7	7	7
33. Does it use any risk prioritization matrix based on severity and frequency criteria?	7	7	7	7	29. No adjustment	7	7	7	7
34. Does it assess the adequacy of risk control or reduction measures?	7	7	7	7	Eliminated after group discussion
Risk treatment					Risk treatment
35. Did it implement basic clinical protocols for patient safety?	7	7	7	7	30. No adjustment	7	7	7	7
36. Did it implement action plans in reaction to investigated adverse events?	7	7	7	7	31. No adjustment	7	7	7	7
37. Does the hospital describe the responsible for the implementation for risk reduction actions?	7	7	7	7	Questions adjusted in the 2nd vote to: 32. Does it present a complete action plan (schedule, responsible, resources and indicators) for risk reduction actions?	7	7	7	7
38. Does the hospital describe the implementation schedule for risk reduction actions?	7	7	7	7
39. Does the hospital describe and measure indicators of implementation and effectiveness of risk reduction actions?	6	7	7	7
40. Does the hospital describe the resources needed for care risk reduction actions?	5/7	6/7	6/7	6/7
Risk communication					Risk communication
41. Does the high management receive periodic communication on the activities and results of care risk management?	7	7	7	7	Do stakeholders of questions 33-35 receive periodic communication on the activities and results of care risk management? 33. High Management 34. Intermediary managers and clinical leaders 35. Care professionals	7	7	7	7
42. Do intermediary managers and clinical leaders receive periodic communication on the activities and results of care risk management?	7	7	7	7
43. Do care professionals receive periodic communications on the activities and results of care risk management?	7	7	7	7
44. Is the communication to patients about the adverse events (open disclosure of errors) that occurred standardized through any institutional norms, protocol or policy?	6	7	6	6	36. No adjustment	7	7	7	7
45. Does the hospital have sent external notifications (e.g. NOTIVISA) regularly in the last 12 months?	7	7	7	7	37. Does it perform external notification by the NOTIVISA system monthly?	7	7	7	7
Integration of risk management processes					Integration of risk management processes
46. Did the hospital perform a complete cycle of risk management (identification, analysis, assessment, treatment and monitoring of risk) in the last 12 months?	7	7	7	7	38. Did it perform a complete cycle of risk management (identification, analysis, assessment, treatment and monitoring) or cycle of quality improvement focused on patient safety (PDCA, assessment and improvement cycle) in the last 12 months?	7	7	7	7
47. Did it record the conduction of Root Cause Analyses or the London Protocol in the last 12 months?	6	7	6	7	39. Did the hospital conduct the Root Cause Analyses or the London Protocol in the last 12 months?	7	7	7	7
48. Did it record the conduction of Failure Mode and Effect Analysis (FMEA) in the last 12 months?	6	7	6	6	40. Did it perform the Failure Mode and Effect Analysis (FMEA) in the last 12 months?	7	7	7	7

Dimension	Subdimension	Formulation of subdimension	No. of items
Structure (12 items)	Awareness	Professionals’ involvement in general, aiming the team’s commitment to the risk management in health care.	2
	Accountability	Definition of an organizational system responsible for the risk management assistance and accountability.	3
	Ability	Necessary resource provision to accomplish the risk management. Includes human resources in appropriate quantity and qualification, time for professionals to devote to activities of risk management, financial resources, equipment, and inputs.	4
	Promoting patient safety culture	Periodic assessment, feedback from employee assessments, and intervention in identified weaknesses, to constantly improve the patient safety culture.	3
Process (28 items)	Risk identification	Risk identification shows the manageable risks. Risk identification may be: retrospective , which identifies previous incidents in the institution and their probability of repetition; in real-time , which identifies risks at the critical time when a support incident may occur; and prospective , which identifies risks based on the future possibility of incidents, although they have not yet occurred at the institution.	13
	Risk assessment and analysis	Risk analysis and assessment activities aim to better understand safety problems before implementing risk reduction actions.	4
	Risk treatment	Planning activity and actions to improve patient safety, therefore avoiding, reducing, or transferring the risks.	3
	Risk communication	Constantly communicating the activities of risk management and the risks to managers, health professionals, patients, and external regulatory bodies.	5
	Process Integration	Risk management processes should be performed in a combined method, with or without the use of integration techniques (e.g. 5 whys, FMEA, etc.).	3
Total			40

Faculdade de Saúde Pública da Universidade de São Paulo Avenida Dr. Arnaldo, 715, 01246-904 São Paulo SP Brazil, Tel./Fax: +55 11 3061-7985 - São Paulo - SP - Brazil
E-mail: revsp@usp.br

Acompanhe os números deste periódico no seu leitor de RSS

[1] Correspondence: Zenewton André da Silva Gama Dep. Saúde Coletiva, Campus Universitário Lagoa Nova - UFRN. Av. Sen. Salgado Filho, s/n, CEP 59078-970. Natal/RN (Brasil). E-mail: zasgama@gmail.com