Patient Classification System for alcohol and other drugs: construction and validation

ABSTRACT

Objective:

To construct and validate a nursing care dependency classification system for alcohol and other drugs.

Method:

A psychometric study. We used the agreement percentage and intraclass correlation coefficient for inter-rater agreement. The Receiver Operating Characteristic was used to determine values and accuracy of the cut-off points. The steps were literature review, content validation and clinical validation.

Results:

Fifteen (15) judges, 6 nurses and 65 users of psychiatric emergency and hospitalization units participated. There were 11 indicators divided into three subcategories of dependency, and a sum between 11 and 33. The agreement regarding the judges’ evaluation on the categories was satisfactory, being equal to or higher than 80%. Agreement was satisfactory among the nurses in most of the indicators (K>0.400), and the Intraclass Correlation Coefficient was equal to 0.723.

Conclusion:

The Classification System of Patients for Alcohol and other Drugs presents statistical evidence of reliability, obtained by a satisfactory inter-rater agreement. Its applicability is widely believed to assist in the management of nursing care and team sizing.

DESCRIPTORS
Inpatients; Substance-Related Disorders; Classification; Nursing Assessment; Nursing Staff; Validation Studies

RESUMO

Objetivo:

Construir e validar um sistema de classificação de dependência de cuidados de enfermagem em álcool e outras drogas.

Método:

Estudo psicométrico. Utilizou-se do percentual de concordância e coeficiente de correlação intraclasse para concordância interavaliadores. Empregou-se o Receiver Operating Characteristic para determinar valores e precisão dos pontos de cortes. As etapas foram revisão de literatura, validação de conteúdo e validação clínica.

Resultado:

Participaram 15 juízes, seis enfermeiros e 65 usuários de emergência psiquiátrica e unidade de internação. Obtiveram-se 11 indicadores, divididos em três subcategorias de dependência, e somatório entre 11 e 33. Em relação à avaliação dos juízes sobre as categorias, a concordância foi satisfatória, sendo igual ou maior a 80%. Entre os enfermeiros, a concordância foi satisfatória na maioria dos indicadores (K>0,400) e Coeficiente de Correlação Intraclasse igual a 0,723.

Conclusão:

O Sistema de Classificação de Pacientes em Álcool e outras Drogas apresenta evidência estatística de confiabilidade, obtida pela satisfatória concordância interavaliadores. Acredita-se amplamente em sua aplicabilidade para auxiliar no gerenciamento da assistência de enfermagem e dimensionamento de equipe.

DESCRITORES
Pacientes Internados; Transtornos Relacionados ao Uso de Substâncias; Classificação; Avaliação em Enfermagem; Recursos Humanos de Enfermagem; Estudos de Validação

RESUMEN

Objetivo:

Construir y evaluar el sistema de clasificación de dependencia de cuidados de enfermería en alcohol y otras drogas.

Método:

Estudio psicométrico. Se utilizó el porcentual de concordancia y coeficiente de correlación intraclase para concordancia interevaluadores. Se empleó el Receiver Operating Characteristic para determinar valores y precisión de los puntos de corte. Las etapas fueron revisión de literatura, validación de contenido y validación clínica.

Resultado:

Participaron 15 jueces, seis enfermeros y 65 usuarios de emergencia psiquiátrica y unidad de estancia hospitalaria. Se lograron 11 indicadores, divididos en tres subcategorías de adicción, y sumatorio entre 11 y 33. Con relación a la evaluación de los jueces acerca de las categorías, la concordancia fue satisfactoria, siendo igual o mayor que el 80%. Entre los enfermeros, la concordancia fue satisfactoria en la mayoría de los indicadores (K>0,400) y Coeficiente de Correlación Intraclase igual que 0,723.

Conclusión:

El Sistema de Clasificación de Pacientes en Alcohol y otras Drogas presenta evidencia estadística de confiabilidad, obtenida por la satisfactoria concordancia interevaluadores. Se cree ampliamente en su aplicabilidad para auxiliar la gestión de la asistencia enfermera y dimensionamiento de equipo.

DESCRIPTORES
Pacientes Internos; Trastornos Relacionados con Sustancias; Clasificación; Evaluación en Enfermería; Personal de Enfermería; Estudios de Validación

INTRODUCTION

The consumption of psychoactive substances in Brazil and in the world is increasing, and its consequences are among the most frequent threats to the health of the population⁽¹⁾. These mainly include traffic accidents, violence, criminality, work absenteeism, reduced performance in professional activities, difficulties in interpersonal relationships, psychological disorders, homicides, suicides, clinical diseases, and hospital admissions; moreover, such consumption represents a risk factor for morbidity and mortality among young people and adults⁽²⁾.

Treatment of dependent users of psychoactive substances aims to evaluate comorbidities and to stabilize crisis situations, with a posterior approach to psychosocial problems and requires adequate permanence time for its effectiveness^(1,3). Considering the dynamics and the susceptibility of motivational stages, users may be ambivalent and reluctant towards the actions inherent to the treatment⁽⁴⁾, thus requiring different amounts of nursing care⁽⁵⁾ which implies the need for technical and scientific training of these professionals to meet these and other demands.

The Federal Nursing Council (Conselho Federal de Enfermagem)⁽⁶⁾ emphasizes that nurses must rigorously assess health needs and plan interventions based on various parameters to ensure comprehensive and continuous care. It also points out that the number of nursing professionals directly interferes with the safety and quality of care. It defines parameters for dimensioning the nursing professional staff in order to subsidize the managers and nurses in decision making about the work process based on the Patient Classification System (PCS) (Sistema de Classificação de Pacientes – SCP)⁽⁶⁾.

Managing professionals in health services is based on labor standards⁽⁷⁾ with a focus on attributes and competencies, but little consideration is given to the variability inherent in care activities and clinical aspects⁽⁸⁾. When there is numerical and/or qualitative insufficiency in the nursing team, injury risks to users and staff increase. Thus, a PCS contributes to establishing the amount of professionals by providing necessary work hours and reducing attendance-related iatrogenies⁽⁹⁾. Classifying the needs of the individual to be assisted is the first step in the nursing personnel dimensioning process, and an important administrative tool to define the complexity of care dependence in relation to the team, and can be applied in different contexts⁽⁵⁾.

The creation of a PCS for alcohol and other drugs is justified because of the inexistence of an instrument that evaluates the dependency of nursing care which targets users of psychoactive substances in specialized service for treatment. This PCS aims to subsidize the daily systematic evaluation performed by nurses about the nursing care needs presented by users addicted to a psychoactive substance in specialized treatment service. Building and validating this PCS will contribute to determining staff dimensioning and will enable identifying complex and multifaceted conditions among users addicted to different psychoactive substances. Thus, this study aimed to construct and validate a Patient Classification System (PCS) for alcohol and other drugs.

METHOD

This study was constituted from three stages. The first was constructing the classification system of dependence in nursing care for alcohol and other drugs based on literature review, having adults of both genders as the target population for its application. The psychometric methodology was used for its construction⁽¹⁰⁾, along with the complexity of care classification model⁽¹¹⁾ and the graphic structure according to a psychiatric nursing dependency level classification instrument⁽¹²⁾.

The second step of content validation was conducted in two rounds. Content validation contemplates evaluation and correction by specialists of the phenomena which is intended to be measured. Twenty (20) experts from the area of interest were invited to participate in this step, but five were excluded because they did not give a response in the first or second round. Thus, the committee consisted of 15 expert judges⁽¹³⁾ who were selected according to criteria adapted from Fehring⁽¹⁴⁾ such as experience, specialization, master's degree and/or doctorate in mental health or alcohol and other drugs.

In the first round, the judges evaluated the system consisting of 10 care indicators, judging the relevance of each indicator. After the feedback, the judges’ suggestions were considered and a new indicator was added, making a total of 11 care indicators in the system. In the second round, the same group of judges re-evaluated each indicator according to the relevance, content clarity and complexity level of each subcategory. The percentage agreement was calculated by dividing the number of participants who agreed on an item by the total number of participants, multiplied by 100. A minimum agreement of 90% among the judges was adopted⁽¹⁵⁾.

The third step consisted of the validation and clinical reliability of the system which occurred in two moments: (I) System Application and (II) Statistical analysis of the data. The first one was carried out in two services specialized in treating people addicted to psychoactive substances in the city of São Paulo, from July to October 2016, including a hospitalization unit and a psychiatric emergency department. Scenario diversification was taken into account for enabling a heterogeneous sample and allows to evaluate the subcategories of the system indicators more comprehensively⁽¹⁵⁾. Six nurses who were previously trained to apply the system participated. Each user was submitted to two evaluations performed by different nurses, who were instructed not to communicate during the system application in order to avoid interference in the results. A sample of 65 users was established when considering the number of indicators that compose the system, surpassing the minimum sample of five for each indicator^(16-17).

The second moment was performing data analysis using the Kappa Coefficient, or Cohen's kappa coefficient (K) and the Intraclass Correlation Coefficient (ICC). The first can be defined as an association measure used to describe and test the degree of inter-rater agreement. The use of Kappa made it possible to analyze the evaluation agreement among nurses. This coefficient may vary between 0 and 1, whose interpretations may be K<0 = no match; 0<K≤0.19 = poor agreement; 0.20<K≤0.39 = mild agreement; 0.40<K≤0.59 = moderate agreement; 0.60<K≤0.79 substantial agreement; 0.80<K≤1.00 = almost perfect agreement⁽¹⁸⁾.

The second coefficient is a measure of agreement that measures the intensity of the intraclass association in the same construct, indicates the reproducibility and the reliability of an instrument, in addition to the variability among the nurses’ evaluations in the study. The ICC value also varies between 0 and 1, and the more the value approaches 1, the better the result^(19,20).

Nurses were asked to evaluate the patients according to their clinical judgment and categorize them into levels of care dependency before applying the system. These levels were pre-established taking into account concepts of Minimal, Intermediate and High dependency, which are present in the literature⁽²¹⁾. When considering such results as the gold-standard, the Youden index⁽²²⁾ was applied in order to obtain the lowest possible proportion of errors in the classification. This index determines the precision of the cut-off points, and the more its value approaches 1, the more positive its interpretation is⁽²²⁾.

The ROC (Receiver Operating Characteristic) curve method was also implemented, which reveals the measure of the ability to correctly classify some given data. This method was used because it is an important tool to establish cut-off points⁽²³⁾ and allows to evaluate how well a condition can be correctly identified or the precision of a test, and can be interpreted as excellent (>0.9), good (0,8-0.9), regular (0.7-0.8), poor (0.6-0.7) or inadequate (0.5-0.6)⁽²³⁾. The database was built in an Excel spreadsheet, and statistical analysis was performed in SPSS Statistics Base 22.0 software. All ethical aspects have been respected and approved in accordance with Opinion No. 105.395/2015.

As a limitation in the study we highlight the fact that the experimental application of the Patient Classification System for Alcohol and Other Drugs (In Portuguese: Sistema de Classificação de Pacientes em Álcool e outras Drogas – SiCAD) was carried out in a hospitalization and psychiatric emergency unit. Nevertheless, the possibility that this could be tested in other scenarios such as a psychosocial attention center is appreciated. Also, it should be pointed out that its application did not occur simultaneously in both places, which made it impossible to obtain nurses’ immediate opinions on the experience of using the system. However, it was possible to circumvent this limitation by asking the nurses for concise feedback, which occurred after the data collection and analysis was finalized. It was felt that such feedback could have been more detailed if it had occurred in a more opportune moment.

RESULTS

The Patient Classification System for Alcohol and Other Drugs (SiCAD) includes 11 indicators with three subcategories (minimum, intermediate and high dependency), with a score ranging from 1 to 3 and a sum from 11 to 33, according to the complexity. Its indicators are: General Aspects; Orientation, Thinking and Sensoperception; Mood and Affection; Attitude and Posture; Behavior; Food and Hydration; Eliminations; Sleep Pattern; Pharmacological and Non-pharmacological treatment; Clinical Aspects and Motivational Aspects (this one was added after the second round).

The minimum dependency classification comprises clinically and psychologically stable users being capable of meeting most of their needs, and therefore being lowly dependent on the nursing staff. In the intermediate dependency classification, users have periods of clinical and/or psychological instability, which requires specific interventions to meet their needs and are partially dependent on the nursing team. Those classified as highly dependent are clinically and/or psychologically unstable, present risks to themselves or others, and are largely dependent on interventions to meet their needs.

In the construct validation, 53% (n=8) of the 15 judges who participated in the study graduated between 7 and 14 years ago, and 47% (n=7) more than 18 years ago; they had postgraduate degrees in mental health, psychiatry or alcohol and other drugs, and there were 14 specialists (93%), 12 masters (67%) and three doctors (20%) among them from different parts of the country. The judges were asked about their agreement on the pertinence of the data that composed the header and the footer only in the first round, because the data obtained were satisfactory. The space designated to be filled with information about users’ “Age” and “Gender”, “Total system points” and “Nurse's signature” was 100%, and 93.3% (n=14) considered the “Date” and “User registration” fields to be relevant. The judges’ assessment of the “Classification” of users was 86.7% (n=13).

The judges also evaluated the relevance of the 10 indicators in the same round, and there was full agreement on the indicators “mood and affection”, “eliminations”, “sleep pattern”, “pharmacological and non-pharmacological aspects” and “clinical aspects”. Disagreement indicators of 6.7% (n=1) were “general aspects”, “orientation, thinking and sensoperception”, “feeding and hydration”, and 13.3% (n=2) with “behavior” and “interaction”.

The judges questioned adopted terms and concepts, and the established interval for degrees of care dependency and clarity in the description of subcategories. After analyzing these data, changes were made in SiCAD and then submitted to the second round in which there was full agreement on the relevance of the indicators, according to Figure 1. Agreement regarding content clarity and the complexity level of each subcategory was over 80%.

Figure 1
Percentage of inter-rater agreement among judges regarding the pertinence of the first and second round indicators – São Paulo, SP, Brazil, 2016.

1-General aspects; 2-Orientation, thinking and sensoperception; 3-Mood and affection; 4- Attitude and posture; 5- Behavior; 6- Feeding and hydration; 7- Eliminations; 8- Sleeping pattern; 9-Pharmacological and non-pharmacological aspects; 10- Clinical aspects; 11-Motivational aspects.

Six nurses participated in the clinical validation stage with a mean age of 37 (ranging from 29 to 58 years old), 5 years of Nursing practice, 3 years of experience with alcohol and drug users/mental health, and all of them with a postgraduate degree in at least one of the two specializations.

Of the participating users, 90.7% (n=59) were males aged between 26 and 59 (76.9%, n=50), considered themselves predominantly white (46%, n=30) and brown (43%, n=28), and catholic (58%, n=36). About 65% (n=42) were single, with secondary (31.8%, n=20) and high school education (57%, n=37), and were unemployed (55.7%, n=34). It was found that 83% (n=53) had been hospitalized before, 24% (n=15) had some psychiatric comorbidity, and 17% (n=11), some clinical comorbidity, of which 9.5% (n=6) were hypertension and 5% (n=3) diabetes mellitus. Regarding the consumption of psychoactive substances, 68% (n=41) were tobacco smokers, 79.7% (n=51) were frequent users of alcohol, 65.6% (n=42) of cocaine, 34.4% (n=22) of crack, and 31% (n=20) marijuana.

The application time of SiCAD was between 10 and 20 minutes, reducible according to the professional's familiarity with the system. The agreement analysis among the nurses showed variation among the indicators. The majority of the indicators obtained an agreement percentage over 80%. Kappa ranged from 0.160 to 0.702, showing that there was statistical significance in most of the indicators, according to Table 1. The result of the Intraclass Correlation Coefficient was 0.723, indicating reproducibility of SiCAD in clinical practice.

Table 1
Agreement percentage distribution and Kappa coefficient for the indicators based on the nurses’ evaluations – São Paulo, SP, Brazil, 2016.

Indicators	Agreement		Kappa
	N	%
1 – General Aspects	62	95.4	0.702
2 – Orientation, Thinking and Sensoperception	52	80.0	0.310
3 – Mood and Affection	52	80.0	0.419
4 – Attitude and Posture	54	83.1	0.469
5 – Behavior	58	89.2	0.662
6 – Food and Hydration	60	92.3	0.339
7 – Eliminations	59	90.7	0.359
8 – Sleep and Rest	55	84.6	0.674
9 – Pharmacological and Non-pharmacological treatment	43	66.1	0.160
10 – Motivational Aspects	46	70.8	0.278
11 – Clinical Aspects	61	93.8	0.609

• K<0 = no agreement; 0<K≤0.19 = poor agreement; 0.20 <K≤0.39 = mild agreement; 0.40<K≤0.59 = moderate agreement; 0.60 <K≤0.79 substantial agreement; 0.80<K≤1.00 = almost perfect agreement.

The complexity classification intervals of Minimum (11 – 13), Intermediate (14 – 20) and High (21 – 33) Dependency were determined by the Youden index⁽²²⁾ based on the nurses’ subjective evaluations, with 82% precision on the cut-off points. Sensitivity values were obtained for each complexity classification from the cross-correlation between the nurses’ evaluations and the total SiCAD score: 83.2% for minimal dependency, 73.3% for intermediate and 100% for high dependency. The Quality of Adjustment Index (ROC) was used in order to validate the reliability of the determined cut-off points, resulting in 0.922.

DISCUSSION

The needs of nursing care, users of alcohol and other drugs, and adults of both genders⁽²⁴⁾ were considered in the SiCAD construction process, ranging from physiological needs to complex psychological aspects and motivational states. Dividing the indicators into subcategories allows for identifying the dependency level that the users present in relation to the nursing team and reflects the stage which they are in, as well as evidencing clinical and psychosocial manifestations⁽²⁵⁾. The indicators that make up the SiCAD are able to provide data on the users’ demands in a comprehensive way and enable planning of nursing care.

The literature identifies the participation of a group of judges as being essential in construct evaluation⁽²⁶⁾. In this study it is recognized that the participating judges presented maturity in the area of alcohol and other drugs, allowing for a rigorous analysis of the indicators. In addition, they contributed to content adequacy when they suggested including a new indicator. Content validation may be limited by judges’ subjectivity, thus an agreement percentage greater than 80% was considered acceptable in this study⁽¹⁵⁾.

When conducting the two rounds to validate the SiCAD content, an evaluation of the relevance of the concepts involved in each indicator and the extent to which they contributed to achieving the objectives proposed by the system⁽²⁷⁾ was considered important, in addition to the judges’ suggestions for qualitatively improving the indicators. After applying the agreement percentage in the results of the questions with dichotomous answers (yes/no), it was possible to systematize them as to the degree of agreement between the judges’ evaluations, and these results were enough to reach the objectives proposed for the content validation step⁽¹⁵⁾.

After SiCAD application, the nurses revealed having familiarity with some indicators since they had already worked with evaluation systems of specific complexity in mental health in existing clinics. The indicators that showed substantial agreement based on Kappa were: 11 – Clinical Aspects; and 1 – General Aspects, which include clinical and/or psychiatric comorbidities, signs and symptoms resulting from the use/abstinence of psychoactive substances, and appearance and fragilities in self-care, respectively. It is understood that these are common evaluations of the nurses’ clinical practice.

The indicators 9 – Pharmacological/Non-pharmacological treatment and 10 – Motivational Aspects, explore drug characteristics, motivational stages, relapse risks, psychosocial aspects and adherence to treatment, and were the indicators that presented less agreement among nurses, but were well evaluated by the judges on their clarity and the increasing level of complexity. It should be emphasized that for an accurate evaluation of the user in relation to these indicators, it is fundamental that the nurses have technical-scientific knowledge and refined skills to recognize the motivational stages that imply the users’ response to treatment⁽²⁶⁾. The need for a more in-depth analysis of the interpretation of these indicators from the nurses’ perspective was then identified in order to understand the possible difficulties they had at the time applying SiCAD.

When using the Kappa coefficient to measure agreement among nurses’ evaluations, it was possible to identify some indicators with mild agreement⁽¹⁾, which is understandable when considering the differences in their training and their professional experience, as well as their familiarity in the use of ranking scales. Nevertheless, a satisfactory ICC value was obtained⁽¹⁹⁾, thus demonstrating that the SiCAD is reliable and reproducible in nursing practice. The cut-off points and intervals were statistically well-defined and support a consistent classification according to the complexity level of nursing care.

The implications for using SiCAD in nurses’ clinical practice can be demonstrated both from the viewpoint of care and management. In the first point, using SiCAD allows for evaluating the real care needs of the users and favors elaborating the evolution and implementation of the Systematization of Nursing Care (SNC). In the second point, as a management tool it maps the population profile that was provided care in order to lead organizational and/or adaptations changes, it identifies weaknesses and points for improvements in order to direct the teams’ actions of continuing education. In addition, from a longitudinal perspective it identifies the complexity of the required care and sustains discussions regarding the quantitative aspect of human resources.

CONCLUSION

This study enabled constructing and validating a Classification System of Care Dependence for Alcohol and Other Drugs, abbreviated as SiCAD. It presents psychometric indicators with acceptable content validity, as well as reliable statistical evidence obtained by satisfactory inter-rater agreement. The classification system intervals are reliable and accurate since they have been statistically adjusted, which is an essential differential and makes them more sensitive to correctly categorize user needs.

SiCAD application is private to the nurse and enables daily care need assessment of users who are dependent on psychoactive substances inserted in a specialized service for treatment. The system shows the attended population's profile and contributes to the adequacy of nursing team dimensioning.

SiCAD proved to be easy to use, applicable in a short period of time and encompasses several aspects that are the focus of nursing care. It is also recognized that its effectiveness can be evidenced from its application in clinical practice. We believe in its importance, emphasizing studies that deepen the use of this technology in nursing care.

REFERÊNCIAS

• ¹
  Meirelles JACM, Chohfi MCF, Basílio MM, Geraldini A, Araújo Filho GM, Laranjeira RR, et al. Nível de adesão ao tratamento ambulatorial de pacientes dependentes de substâncias psicoativas. Espácios [Internet]. 2015 [citado 2017 abr. 20];36(4). Disponível em: http://www.revistaespacios.com/a15v36n04/15360412.html
  » http://www.revistaespacios.com/a15v36n04/15360412.html
• ²
  Pereira MO, Vargas D, Oliveira MAF. Reflexão acerca da política do Ministério da Saúde brasileiro para a atenção aos usuários de álcool e outras drogas sob a óptica da sociologia das ausências e das emergências. SMAD Rev Eletr Saúde Mental Álcool Drog [Internet]. 2012 [citado 2017 mar. 15];8(1):9-16. Disponível em: http://www.revistas.usp.br/smad/article/view/49597
  » http://www.revistas.usp.br/smad/article/view/49597
• ³
  National Institute on Drug Abuse. Principles of drug addiction treatment: a research base-guide [Internet]. Bethesda: NIH; 2014 [cited 2017 Apr 20]. Available from: https://www.drugabuse.gov/publications/principles-drug-addiction-treatment-research-based-guide-third-edition/principles-effective-treatment
  » https://www.drugabuse.gov/publications/principles-drug-addiction-treatment-research-based-guide-third-edition/principles-effective-treatment
• ⁴
  Barbosa NL, Mangueira SO, Albuquerque JG, Guimarães FJ. Cuidado de enfermagem a pacientes alcoolistas: percepções da equipe de enfermagem. Rev Bras Pesq Saúde. 2013; 15(2):88-93. DOI: http://dx.doi.org/10.21722/rbps.v0i0.5679
  » http://dx.doi.org/10.21722/rbps.v0i0.5679
• ⁵
  Oliveira RP, Laus AM. Characterization of patients in psychiatric hospitalization units according to the degree of dependence on nursing care. Rev Esc Enferm USP. 2011;45(5):1160-6. DOI: http://dx.doi.org/10.1590/S0080-62342011000500019
  » http://dx.doi.org/10.1590/S0080-62342011000500019
• ⁶
  Conselho Federal de Enfermagem. Resolução n. 0527/2016. Parâmetros para o dimensionamento do quando de profissionais de enfermagem [Internet]. Brasília: COFEn; 2016 [citado 19 nov. 2016]. Disponível em: http://www.cofen.gov.br/resolucao-cofen-no-05272016_46348.html
  » http://www.cofen.gov.br/resolucao-cofen-no-05272016_46348.html
• ⁷
  Rossetti AC, Gaidzinski RR, Bracco MM. Determinação da carga de trabalho e do dimensionamento da equipe de enfermagem em um pronto-socorro pediátrico. Einstein (São Paulo) [Internet]. 2014 [citado 2017 abr. 27];12(2):217-22. Disponível em: http://www.scielo.br/pdf/eins/v12n2/1679-4508-eins-12-2-0217.pdf
  » http://www.scielo.br/pdf/eins/v12n2/1679-4508-eins-12-2-0217.pdf
• ⁸
  Dias KCF, Chiwiacowsky LD, Korzenowsk AL. Otimização de quadro de lotação hospitalar: aplicação da meta-heurística Busca Tabu. Espácios [Internet]. 2015 [citado 2017 abr. 20];36 (21). Disponível em: http://www.revistaespacios.com/a15v36n21/15362108.html
  » http://www.revistaespacios.com/a15v36n21/15362108.html
• ⁹
  Fugulin FMT, Oliveira JLC, Nicola AL, Araújo ASS, Marinho AM, Canavezi CM, et al. Dimensionamento de profissionais de enfermagem: implicações para a prática assistencial. Divulg Saúde Debate. 2016;(56):126-33.
• ¹⁰
  Pasquali L. Psychometrics. Rev Esc Enferm USP [Internet]. 2009 [cited 2017 Apr 13]; 43(n.spe):992-9. Available from: http://www.scielo.br/pdf/reeusp/v43nspe/en_a02v43ns.pdf
  » http://www.scielo.br/pdf/reeusp/v43nspe/en_a02v43ns.pdf
• ¹¹
  Perroca MG. Development and content validity of the new version of a Patient Classification Instrument. Rev Latino Am Enfermagem [Internet]. 2011 [cited 2017 Apr 26];19(1):58-66. Available from: http://www.scielo.br/pdf/rlae/v19n1/09.pdf
  » http://www.scielo.br/pdf/rlae/v19n1/09.pdf
• ¹²
  Martins PASF, Arantes EC, Forcella HT. Patient classification system in psychiatric nursing: clinical validation. Rev Esc Enferm USP [Internet]. 2008 [cited 2017 Apr 26];42(2):233-41. Available from: http://www.scielo.br/pdf/reeusp/v42n2/a03.pdf
  » http://www.scielo.br/pdf/reeusp/v42n2/a03.pdf
• ¹³
  Borsa JC, Damásio BF, Bandeira DR. Adaptação e validação de instrumentos psicológicos entre culturas: algumas considerações. Paidéia (Ribeirão Preto) [Internet]. 2012 [citado 2017 ago. 30];22(53):423-32. Disponível em: http://www.scielo.br/pdf/paideia/v22n53/14.pdf
  » http://www.scielo.br/pdf/paideia/v22n53/14.pdf
• ¹⁴
  Melo RP, Moreira RP, Fontenele FC, Aguiar ASC, Joventino ES, Carvalho EC. Critérios de seleção de experts para estudos de validação de fenômenos de enfermagem. Rev Rene [Internet]. 2011 [citado 2017 mar. 20];12(2):424-31. Disponível em: http://www.revistarene.ufc.br/revista/index.php/revista/article/view/174
  » http://www.revistarene.ufc.br/revista/index.php/revista/article/view/174
• ¹⁵
  Alexandre NMC, Coluci MZO. Validade de conteúdo nos processos de construção e adaptação de instrumentos de medidas. Ciênc Saúde Coletiva [Internet]. 2011 [citado 2017 abr. 26];16(7):3061-8. Disponível em: http://www.scielo.br/pdf/csc/v16n7/06.pdf
  » http://www.scielo.br/pdf/csc/v16n7/06.pdf
• ¹⁶
  Kline RB. Principles and practice of Structural Equation Modeling. 3rd edition. New York: Guilford Press; 2011.
• ¹⁷
  Marôco J. Análise de equações estruturais: fundamentos teóricos, software e aplicações. Pêro Pinheiro: Report Number; 2010.
• ¹⁸
  Dini AP, Fugulin FMT, Veríssimo MLÓR, Guirardello EB. Pediatric Patient Classification System: construction and validation of care categories. Rev Esc Enferm USP. 2011;45(3):574-9. DOI: http://dx.doi.org/10.1590/S0080-62342011000300004
  » http://dx.doi.org/10.1590/S0080-62342011000300004
• ¹⁹
  Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979;86(2):420-8.
• ²⁰
  Matos DAS. Confiabilidade e concordância entre juízes: aplicações na área educacional. Est Aval Educ. 2014;25(59):298-324.
• ²¹
  Perroca MG, Gaidzinski RR. Avaliando a confiabilidade interavaliadores de um instrumento para classificação de pacientes: coeficiente Kappa. Rev Esc Enferm USP. 2003;37(1):72-80. DOI: http://dx.doi.org/10.1590/S0080-62342003000100009
  » http://dx.doi.org/10.1590/S0080-62342003000100009
• ²²
  Kendall MG. The advanced theory of statistic. London: Charles Griffin; 1949.
• ²³
  Tang K, Wang R, Chen T. Towards maximizing the area under the ROC curve for multi-class classification problems [Internet]. San Francisco, CA, USA; 2011 [cited 2017 Apr 26]. Available from: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.395.901&rep=rep1&type=pdf
  » http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.395.901&rep=rep1&type=pdf
• ²⁴
  Silva AB. Cuidados de enfermagem a pacientes com dependência química. REBES Rev Bras Educ Saúde [Internet]. 2015 [citado 2017 abr. 26];5(1):1-5. Disponível em: http://www.gvaa.com.br/revista/index.php/REBES/article/view/3145/2640
  » http://www.gvaa.com.br/revista/index.php/REBES/article/view/3145/2640
• ²⁵
  Henriques JAS, Hildebrandt LM, Leite MT, Van der Sand ICP. Cuidado a pessoas com dependência química em hospital geral na ótica da equipe de enfermagem. Rev Enferm UFSM [Internet]. 2013 [citado 2017 abr. 26];3(3):383-93. Disponível em: https://periodicos.ufsm.br/reufsm/article/view/7998
  » https://periodicos.ufsm.br/reufsm/article/view/7998
• ²⁶
  McGilton K. Development and psychometric evaluation of Supportive Leadership Scales. J Nurs Scholarsh. 2010;42(2):223-32. DOI: 10.1111/j.1547-5069..2009.01323.x
  » https://doi.org/10.1111/j.1547-5069..2009.01323.x
• ²⁷
  Ferreira ACZ, Capistrano FC, Souza EB, Borba LO, Kalinke LP, Maftum MA. Drug addicts treatment motivations: perception of family members. Rev Bras Enferm [Internet]. 2015 [cited 2017 Apr 26]; 68(3):474-81. Available from: http://www.scielo.br/pdf/reben/v68n3/en_0034-7167-reben-68-03-0474.pdf
  » http://www.scielo.br/pdf/reben/v68n3/en_0034-7167-reben-68-03-0474.pdf

Publication Dates

• Publication in this collection
  2018

History

• Received
  20 June 2017
• Accepted
  19 Dec 2017