Development and content validation of a risk classification instrument

The study included two stages: item generation and content validity. A literature review and retrospective analysis of medical records were conducted to create the instrument items. The Content Validity Ratio (CVR) was used to assess agreement among judges during content validation.

Results:

In the first and second rounds, 75 and 71 judges validated the risk classification instrument, respectively. The minimum adherence score for the latent variable item based on the final number of judges was 0.22 and 0.18; thus, 52 items, divided into three classification categories (red, orange, and yellow), were retained.

Conclusion:

The instrument was considered valid regarding clarity, relevance, pertinence, and agreement regarding the severity indicated in the item.

Descriptors:
Triage; Validation Studies; Psychometrics; Patient Acuity; Primary Health Care

Characterization Data	First Round (n = 75)	Second Round (n = 71)
Mean age in years (SD)	39.95 (8.00)	42 (8.00)
Categorical variables: no (%)
Sex
Female	54 (72.00)	48 (67.60)
Male	21 (28.00)	23 (32.40)
Profession
Physician	15 (20.00)	11 (15.50)
Nurse	60 (80.00)	60 (84.50)
Highest educational level
Undergraduate degree	8 (10.67)	9 (12.70)
Residency	4 (5.33)	2 (2.80)
Specialization	49 (65.33)	43 (60.60)
Master’s degree	6 (8.00)	12 (16.90)
Doctorate	6 (8.00)	5 (7.00)
Post-doctorate	2 (2.67%)	0 (0.00)
Brazilian region of professional activity
North	1 (1.33)	1 (1.40)
Northeast	1 (1.33)	2 (2.80)
Midwest	6 (8.00)	2 (2.80)
South	11(14.67)	10 (14.10)
Southeast	56 (74.67)	56 (78.90)
Area of practice or research^* *It was possible to select more than one option; SD – standard deviation.
Urgency and Emergency	38 (50.67)	42 (59.20)
Primary Care	40 (53.33)	29 (40.80)
Nursing Care in a hospital setting	16 (21.33)	24 (33.80)
Obstetrics	1 (1.33)	2 (2.8)
Pediatrics	12 (16.00)	5 (7.0)

Risk Classification Instrument Items	Clarity (CVR)	Relevance (CVR)	Pertinence (CVR)	Concordance with Classification (%)
Red Category
More than 25% of body surface area burned	0.41	−0.07	−0.17	98.70
Traumatic amputation	0.68	−0.17	−0.23	98.70
Baby being born	0.73	−0.33	−0.23	98.70
Spinal column compromise	0.41	−0.12	−0.52	97.30
Convulsing	0.87	−0.23	−0.25	100.00
Unresponsive child	0.79	−0.17	−0.33	98.70
Open fracture of large parts	0.68	−0.15	−0.28	100.00
Witnessed gastrointestinal hemorrhage	0.55	−0.17	−0.28	100.00
Otorrhagia (Battle’s sign – Raccoon eyes)	0.49	−0.12	−0.31	100.00
Facial burns involving airways	0.73	−0.09	−0.23	100.00
Second- and third-degree burns in the perineal area	0.63	−0.20	−0.23	100.00
Local burns with exposure of systems or organs	0.65	−0.17	−0.28	100.00
Ineffective breathing	0.55	−0.17	−0.28	96.00
Uncontrollable bleeding	0.65	−0.15	−0.28	98.70
Severe trauma	0.49	−0.12	−0.25	100.00
Non-patent airway	0.71	−0.15	−0.31	100.00
Orange Category
Bulging fontanelle	0.68	−0.07	−0.20	96.00
Psychomotor agitation	0.60	−0.09	−0.12	98.70
Sudden visual alteration	0.57	−0.07	−0.33	96.00
Neurological changes within 24 hours	0.55	−0.15	−0.28	96.00
Amputation of small parts	0.60	−0.04	−0.12	94.70
Avulsion of permanent tooth within two hours	0.36	−0.23	−0.31	93.30
Bladder distension	0.55	−0.20	−0.12	93.30
Prostrate child	0.52	−0.12	−0.15	97.30
Evident respiratory distress	0.60	−0.01	−0.20	92.00
Decreased fetal movements	0.60	−0.17	−0.12	94.70
Precordial pain	0.63	−0.04	−0.31	89.30
Enterorrhagia	0.63	−0.09	−0.25	98.70
Between 24% and 15% of body surface area burned	0.55	−0.12	−0.20	96.00
Witnessed epistaxis	0.65	−0.12	−0.07	98.70
Postictal state	0.41	−0.07	−0.28	96.00
Vaginal bleeding in pregnant women over 20 weeks	0.60	−0.20	−0.20	96.00
Intense vaginal bleeding	0.39	−0.25	−0.12	88.00
Small part fracture	0.49	−0.09	−0.36	97.30
Ketotic breath	0.71	−0.09	−0.20	97.30
Inhalation of chemicals	0.63	−0.04	−0.36	97.30
Smoke inhalation	0.68	0.01	−0.33	96.00
Insertion of potentially dangerous object	0.52	−0.17	−0.23	93.30
Pelvic instability	0.49	−0.04	−0.39	89.30
Alcohol intoxication	0.55	−0.09	−0.23	97.30
Drug intoxication	0.57	−0.09	−0.28	97.30
Chemical intoxication	0.60	−0.12	−0.31	96.00
Lacerating injury	0.47	−0.01	−0.25	93.30
Chemical injury	0.55	−0.01	−0.25	96.00
Known lethal venom injury	0.60	−0.01	−0.41	88.00
Arterial or venous vascular injury	0.55	−0.12	−0.39	94.70
Melena	0.60	−0.15	−0.33	98.70
Meningism	0.60	−0.04	−0.15	96.00
Severe ear pain	0.63	−0.01	−0.20	96.00
Amniotic fluid loss	0.55	−0.25	−0.20	94.70
Associated precordial pain	0.41	−0.12	−0.28	89.30
Priapism	0.36	−0.20	−0.33	92.00
Burn with adherent objects to the skin	0.71	−0.09	−0.15	94.70
Facial burn	0.71	−0.15	−0.12	92.00
Non-compressible bleeding	0.23	−0.07	−0.33	96.00
Withdrawal signs	0.49	−0.04	−0.20	96.00
Severe dehydration signs	0.57	−0.15	−0.15	93.30
Acute testicular torsion	0.57	−0.15	−0.52	96.00
Labor	0.63	−0.17	−0.07	93.30
Closed abdominal trauma	0.47	−0.07	−0.33	89.30
Trauma with deformities	0.55	−0.04	−0.23	92.00
Trauma with penetrating objects in vital regions	0.65	−0.04	−0.33	84.00
Yellow Category
Less than 15% of body surface area burned (second-degree)	0.60	−0.17	−0.23	96.00
Tonsillar abscess	0.55	−0.12	−0.20	96.00
Severe and acute test alterations	0.49	−0.09	−0.15	96.00
Avulsion of permanent tooth over two hours	0.55	−0.15	−0.23	93.30
Intermittent crying in a child under 2 years	0.57	−0.20	−0.23	89.30
Deformity of small bones	0.44	−0.12	−0.23	97.30
Jaw dislocation	0.52	−0.12	−0.31	94.70
Shoulder pain in pregnant women (Kehr’s sign)	0.47	−0.15	−0.25	92.00
Ventilation-dependent pain	0.49	−0.09	−0.17	94.70
Apparent mandibular edema	0.44	−0.07	−0.20	97.30
Periorbital edema with inflammatory signs	0.49	−0.09	−0.17	96.00
Significant genital edema	0.63	−0.12	−0.15	88.00
Muscular induration	0.33	−0.07	−0.28	88.00
Between 15% and 9% of body surface area burned (first-degree)	0.49	−0.17	−0.23	98.70
Reported frank hematuria	0.57	−0.12	−0.12	97.30
History of no urination in the last 24 hours	0.60	−0.09	−0.12	96.00
Suicidal ideation	0.60	−0.07	−0.20	98.70
Object insertion	0.47	−0.12	−0.17	96.00
Skin lesions suggestive of infectious diseases (blisters, petechiae)	0.47	−0.12	−0.12	89.30
Electrical injury	0.57	−0.09	−0.31	97.30
Suspected maltreatment injury	0.55	−0.15	−0.12	84.00
Movement limitation	0.39	−0.17	−0.07	93.30
Postictal state	0.41	−0.15	−0.25	98.70
Inflammatory process with serous, aqueous, or purulent exudate	0.63	−0.20	−0.09	96.00
Compressible bleeding	0.60	0.04	−0.17	96.00
Apparent dehydration signs	0.57	−0.12	−0.12	98.70
Inflammatory signs in the breast	0.63	−0.17	−0.04	94.70
Signs suggestive of sexual violence	0.49	−0.15	−0.12	93.30
Cough with hemoptysis	0.52	−0.07	−0.20	96.00
Eye trauma	0.55	−0.15	−0.23	92.00
Assault victim	0.47	−0.17	−0.12	93.30
Intermittent vomiting	0.49	−0.17	−0.12	96.00

Instrument Items	Clarity (CVR)	Relevance (CVR)	Pertinence (CVR)	Concordance with Classification (%)
Red Category
Traumatic amputation	0.97	0.97	0.97	98.60
Baby being born	0.73	0.88	0.94	98.60
Spinal column compromise	0.41	0.91	0.38	97.19
Convulsing	0.87	0.97	0.97	98.60
Unresponsive child	0.79	0.97	1.00	98.60
Open fracture of large parts	0.68	1.00	0.94	97.19
Witnessed gastrointestinal hemorrhage	0.55	0.94	0.83	97.19
Otorrhagia (Battle’s sign - Raccoon eyes)	0.49	0.94	0.88	97.19
Second- and third-degree burns in the perineal area	0.63	0.94	0.94	97.19
Local burns with exposure of systems or organs	0.65	1.00	0.97	98.60
Ineffective breathing	0.55	1.00	1.00	98.60
Active uncontrollable bleeding	0.97	0.97	1.00	97.19
Severe trauma	0.49	1.00	0.88	98.60
Non-patent airway	0.71	0.97	0.85	98.60
Orange Category
Psychomotor agitation with risk of aggression	0.97	0.94	0.94	98.60
Neurological changes within 24 hours	0.55	1.00	0.80	98.60
Decreased fetal movements after one hour of active observation by the mother	0.60	0.88	0.57	98.60
Precordial pain	0.63	1.00	0.91	97.19
Postictal state	0.41	0.88	0.73	98.60
Vaginal bleeding in pregnant women over 20 weeks	0.60	0.94	0.82	98.60
Intense vaginal bleeding	0.39	0.88	0.80	95.78
Small part fracture	0.49	0.83	0.49	98.60
Inhalation of chemicals and/or smoke	0.68	0.97	0.69	98.60
Chemical and/or drug intoxication (licit and illicit)	0.88	0.94	0.74	98.60
Lacerating injury	0.47	1.00	0.94	98.60
Chemical injury	0.55	0.97	0.83	98.60
Severe ear pain with or without secretion	0.88	0.74	0.63	97.19
Associated precordial pain	0.41	0.94	0.83	97.19
Facial burn	0.71	0.97	1.00	97.19
Active non-compressible bleeding	0.74	0.85	0.68	97.19
Signs of alcohol and/or chemical withdrawal	0.91	0.71	0.74	97.19
Signs of meningism	0.74	0.91	0.67	98.60
Signs of closed abdominal trauma	0.88	0.97	0.36	95.78
Severe dehydration signs	0.57	1.00	0.80	97.19
Acute testicular torsion	0.57	0.85	0.21	98.60
Labor	0.97	0.88	0.88	97.19
Trauma with gross deformities	0.85	0.88	0.88	95.78
Trauma with penetrating objects in vital regions	0.97	0.97	1.00	94.37
Yellow Category
Less than 15% of body surface area burned (second-degree)	0.60	0.85	0.91	98.60
Avulsion of permanent tooth over two hours	0.55	0.71	0.80	98.60
Between 15% and 9% of body surface area burned (first-degree)	0.49	0.80	0.91	98.60
Foreign body insertion	0.859	0.80	0.54	98.60
Suspected maltreatment injury	0.55	0.71	0.77	98.60
Acute movement limitation	0.39	0.88	0.82	98.60
Inflammatory process with serous, aqueous, or purulent exudate	0.63	0.60	0.91	97.19
Active compressible bleeding	0.746	0.79	0.91	95.78
Signs of tonsillar abscess	0.831	0.69	0.28	97.19
Inflammatory signs in the breast	0.857	0.68	0.83	98.60
Signs suggestive of sexual abuse	0.915	0.83	0.42	97.19
Signs suggestive of violence	0.859	0.80	0.85	97.19
Eye trauma	0.55	0.91	0.85	95.78
Intermittent vomiting	0.49	0.77	0.77	97.19

Associação Brasileira de Enfermagem SGA Norte Quadra 603 Conj. "B" - Av. L2 Norte 70830-102 Brasília, DF, Brasil, Tel.: (55 61) 3226-0653, Fax: (55 61) 3225-4473 - Brasília - DF - Brazil
E-mail: reben@abennacional.org.br

Acompanhe os números deste periódico no seu leitor de RSS

[1] Corresponding author
Marcia Beatriz Micha Ferreira de Oliveira E-mail: marciamicha.oliveira@gmail.com