Creation of complexity assessment tool for patients receiving home care

Diseño de instrumento de clasificación de complejidad asistencial de pacientes en atención domiciliaria

Abstract

OBJECTIVE

To create and validate a complexity assessment tool for patients receiving home care from a public health service.

METHOD

A diagnostic accuracy study, with estimates for the tool's validity and reliability. Measurements of sensitivity and specificity were considered when producing validity estimates. The resulting tool was used for testing. Assessment by a specialized team of home care professionals was used as the gold standard. In the tool's reliability study, the authors used the Kappa statistic. The tool's sensitivity and specificity were analyzed using various cut-off points.

RESULTS

On the best cut-off point-21-with the gold standard, a sensitivity of 75.5% was obtained, with the limits of confidence interval (95%) at 68.3% and 82.8% and specificity of 53.2%, with the limits of confidence interval (95%) at 43.8% and 62.7%.

CONCLUSION

The tool presented evidence of validity and reliability, possibly helping in service organization at patient admission, care type change, or support during the creation of care plans.

Descriptors
Home Care Services; Home Nursing; Validation Studies; Reproducibility of Results; Sensitivity and Specificity

Resumen

OBJETIVO

Confeccionar y validar instrumento de clasificación de complejidad asistencial de pacientes en atención domiciliaria de un servicio sanitario público.

MÉTODO

Estudio de precisión de diagnóstico, con estimaciones de validez y responsabilidad del instrumento. Para la estimación de la validez fueron consideradas las medidas de sensibilidad y especificidad, tales como la prueba, el instrumento confeccionado y como regla de oro, la clasificación atribuida por equipo especializado de profesionales sanitarios en atención domiciliaria. En el estudio de reproducibilidad del instrumento se empleó la estadística Kappa. La sensibilidad y la especificidad del instrumento fueron analizadas considerándose distintos puntos de corte.

RESULTADOS

Para el mejor punto de corte - 21 - con la regla de oro se logró Sensibilidad del 75,5% con los límites del IC (95%) iguales que el 68,3% y el 82,8% y Especificidad igual que el 53,2% con los límites del IC (95%) iguales que el 43,8% y el 62,7%.

CONCLUSIÓN

El instrumento presentó evidencias de validez y reproducibilidad, pudiendo ayudar la organización del servicio, tanto en el ingreso del paciente como en la migración de modalidad asistencial y el suporte para la confección del plan terapéutico.

Descriptores
Servicios de Atención de Salud a Domicilio; Atención Domiciliaria de Salud; Estudios de Validación; Reproducibilidad de Resultados; Sensibilidad y Especificidad

Resumo

OBJETIVO

Elaborar e validar instrumento de classificação de complexidade assistencial de pacientes em atenção domiciliar de um serviço público de saúde.

MÉTODO

Estudo de acurácia de diagnóstico, com estimativas de validade e de reprodutibilidade do instrumento. Para a estimativa da validade foram consideradas as medidas de sensibilidade e especificidade; como teste, o instrumento elaborado e como padrão ouro, a classificação atribuída por equipe especializada de profissionais de saúde em atenção domiciliar. No estudo de reprodutibilidade do instrumento foi utilizada a estatística Kappa. A sensibilidade e especificidade do instrumento foram analisadas considerando-se diferentes pontos de corte.

RESULTADOS

Para o melhor ponto de corte - 21 - com o padrão ouro obteve-se Sensibilidade de 75,5% com os limites do IC (95%) iguais a 68,3% e 82,8% e Especificidade igual a 53,2% com os limites do IC (95%) iguais a 43,8% e 62,7%.

CONCLUSÃO

O instrumento apresentou evidências de validade e reprodutibilidade, podendo vir a auxiliar na organização do serviço, quer na admissão do paciente, quer na migração de modalidade assistencial e no suporte para elaboração do plano terapêutico.

Descritores
Serviços de Assistência Domiciliar; Assistência Domiciliar; Estudos de Validação; Reprodutibilidade dos Testes; Sensibilidade e Especificidade

Introduction

Home care (HC) in Brazil's public health system is still a very recent type of care, with great potential for implementation and growth, because after studies in trials for home care services were already in place in the country¹, the Ministry of Health launched the Programa Melhor em Casa (Better at Home Program), home care in the realm of the Unified Health System (SUS), in 2011, which is regulated under Ordinance no. 963, of 2013². It reaffirms HC as a technological incorporation that can substitute for or complement hospital interventions of low and medium complexity and care that start at urgency and emergency care services. It can also complement basic care³.

This ministry ordinance defines three types of HC according to patients' clinical profiles, the frequency of home visits, and the composition of the health team responsible for administering care:

HC 1: Users that require low-intensity care, with regular visits from the basic health team

HC 2: Users that require higher intensity care, with at least weekly visits

HC 3: Users that fall under HC 2's criteria with the addition of noninvasive ventilation support, paracentesis, or peritoneal dialysis

HC2 and HC3 patients are the target demographics of Home Care Services under SUS.

Therefore, HC in SUS requires the incorporation of new care technology⁴, which places the challenges of comprehensiveness and continuity of care-both in relation to patients, caregivers, and families and to their care network-in a new noninstitutionalized practice field: the home ^(5-6.

In parallel to this context, current epidemiological tendencies show that the Brazilian population is quickly aging⁷. Increasing operational costs related to medicine's technological development and the need for changes in the current model of technological care (defined by low rationality and efficiency, among other factors), show that HC can be a very consistent alternative of care, especially in relation to hospital stays⁸.

It is very important to consider possibilities for increasing coverage at lower costs and higher effectiveness is very important when it comes to patients, caregivers, and families, as well as in relation to public expenditure and resource management⁹.

The creation of a tool that makes it possible for services to objectively and easily assess the complexity of each case will give health teams better ways to estimate resource use; therefore, it offers a clearer view of admission capacity and better conditions for prioritization.

Analysis of the literature addressing patient assessment tools showed that various tools have been developed in hospital units and, later, in intensive care units and clinics for allocation of nursing personnel, that is, the Patient Assessment System (SPC). This system evaluates the extension of the patients' dependency on nursing care and the assessment of workload, considering the severity variable as directly related to the number of therapeutic interventions and hours of care required^10-13.

When analyzing instruments for the assessment of patients receiving home care in other countries as well as in Brazilian trial runs^14-20, the authors found considerable diversity of models and organizations in home care in the last few years, in addition to various study designs and sample characteristics. However, this analysis does not give a complete understanding of the main aspects of home care in those countries and in those services, especially public HC services, that are defined mainly by their humanized and inclusive nature, with user-centered care plans and multidisciplinary health teams.

The aims of this study were to elaborate and validate a patient assessment tool according to type of care (HC1 or HC2/HC3) for home care services.

Method

This is a diagnostic accuracy study²¹, conducted in the (Home Care Program of the State Health Secretariat of the Federal District (PID-DF).

Tool creation

Monthly workshops were conducted in the period between 2009 to 2011, by professionals who are specialists in the area and who have vast experience with the definition of the main procedures conducted by multi-professional teams. These professionals helped the author in an analysis of time spent on home care by multi-professional teams as a basis for the allocation of personnel ⁽²².

Following this, patients who were registered in the program were profiled according to parameters considered essential for complexity assessment, such as: cardiorespiratory pattern; nutritional status; hemorrhage risk; infections; professionals involved; types of procedures conducted; number of checkups; and degree of efficiency of caregiver/family.

These parameters were based not only on procedures, but on patients' clinical status, on the attributes of the multi-professional health teams (defined here as physician, nursing personnel, nutritionist, and physical therapist), and on the performance of caregivers/families (Table 1).

Table 1
Profile of moderate (HC2) and severe (HC3) patients of the PID-DF, according to the selected criteria - Brasília, Federal District, 2011.

PARAMETERS/PROFILE	MODERATE (STABLE) HC2	SEVERE (POTENTIALLY UNSTABLE) HC3
Cardiorespiratory pattern	Oxygen therapy at home, with possibility of changes in cardiorespiratory pattern	Presence of dyspnea, tachycardia/bradycardia, or reversible arrhythmia during home procedures
Nutritional status	Degree II of malnutrition	Oral route without reaching at least 50% of needs
	Obesity and overweight	Degree III of malnutrition
		Morbid obesity
Hemorrhage risk	Patients with probability of digestive hemorrhage and	Presence of extensive wounds with use of anticoagulants
	continuous use of anticoagulants	Oncological lesions with potential bleeding
		Users of nasoenteric (NET) or gastric tubes (NGT) with a history of ulcers
Presence of infection	Presence of infection with stable clinical status	Presence of acute and recurrent infections, with risk of hemodynamic repercussions
Professionals allocated	Requires at least three professionals	Potentially the entire multidisciplinary team
Types of procedures conducted	Routine procedures, such as:	Need to perform at-home procedures usually performed in hospitals and that require specific care and risks, such as:
	a) cleansing and tracheostomy aspiration	a) Tracheostomy change
	b) Degree II bandages	b) Degree III and IV bandages
	c) Instructions for NET	c) Change from NET to enteral nutrition
	d) Occasional administration of medication via IV, IM, or hypodermoclysis	d) Frequent administration of medication via IV, IM, or hypodermoclysis
		e) Analgesia control
Number of visits	Requires an average of two visits per month	Requires at least two visits per week
Degree of efficiency of caregiver/family	Presence of formal caregiver	Family with frequently changing caregivers
	Family with a good cognitive level and/or motivation to perform what is instructed	Low level of understanding and assimilation of instructions
	Good patient/family and caregiver/team emotional bonds	Bad patient/family emotional bond

• ^HC2
  Care type 2
• ^HC3
  Care type 3

Analyzing the list of the main procedures, the time spent by the health team to perform them, and the selected criteria for patient severity according to the particular aspects of the home care service, the authors found in the Nursing Activities Score (NAS) the necessary outline to format the tool.

The NAS is an internationally validated scale to assess the workload of nurses in quality care in intensive care units with seven domains and 23 activity categories: basic activities (monitoring and controls, hygiene, mobilization, support for patients and family members, and managerial and administrative activities); supports (ventilatory, cardiovascular, renal, neurologic, metabolic); and specific interventions²³.

The biggest difference between NAS and previous tools was the inclusion of activities related to patient status. It was not limited to applied therapeutics and to activities that were excessively time-consuming for professionals; instead, it defined fewer activities in order to be wide-ranging and easily applied to services.

The NAS was adapted by expanding its activities to accommodate a multi-professional team, keeping them in the domain of "basic activities," which was renamed "home activities." The authors eliminated the following activities: "hygiene," because it is conducted by caregivers and/or family members; and "supports" and "specific interventions," because they are part of the ICU environment. The following activities were added: "therapeutic procedures"; "laboratory investigations;" and "rehabilitation."

Thus, the instrument had two domains: "Home activities" and "Managerial and administrative activities" and a total of 20 activities.

Each activity was rewritten and operationally defined according to the main activities selected from the home care service, as well as their profiles; the score was calculated according to measurements of time spent in procedures conducted at home²¹ and the estimated time for administrative and managerial activities. A score of "1" was given to procedures that lasted from 0 to 9 minutes, a score "2" for activities that lasted from 10 to 19 minutes, and so on.

After the group of professionals/specialists reached a consensus on the tool, its first version was finished. It was pre-tested by being applied to 18 medical records of patients registered at the PID-DF, chosen at random. With some adjustments, the final version of the tool was achieved.

Data collection

Inclusion criteria for study participants were the same as admission criteria for the program: patients with chronic degenerative diseases that became acute; patients with sequelae and comorbidities; patients receiving palliative care; patients with functional disabilities for activities of daily living, temporary or permanent; and patients in clinical stability.

Exclusion criteria were: patients requiring invasive mechanical ventilation; those under continuous monitoring; those with intensive nursing and using complex medication with potentially severe collateral effects or with difficult administration.

Thus, the research's reference population consisted of 826 active patients of the PID-DF, Primary Care, Home Care Service, State Secretariat of Health of the Federal District, in the periods of July and August 2012 and June to July 2013, who were empirically defined as HC1, HC2, and HC3-in other words, without scales, based only on the experience and consensus of at least three PID-DF professionals.

For statistical calculation of sample size, the authors initially applied the assessment tool to a pilot sample of 96 patients (2012), aiming to estimate the proportion (sensitivity) who were empirically categorized as HC1, HC2, or HC3.

Considering a level of significance of 5% and a margin of error of 5%, the authors estimated a final random sample of 252 patients, who were categorized by empirical care type and by the coverage area of the PID-DF; the pilot sample was added to the final sample size.

Two PID-DF examiners (Examiner 1 and Examiner 2) were chosen and trained to apply the tool to patients who were selected inside their coverage area, during their work shifts, and without interfering in their service routine.

The criteria chosen to select examiners were: PID-DF professionals only, with the appropriate profile, and available to apply the tool in the homes of patients chosen inside their coverage area, who were either volunteers or appointed and who formally accepted by signing an informed consent.

The following standardization of procedures was applied during the examiners' training, with the goal of correctly applying the tool for statistical validity and reliability:

1. Each examiner scored tool items only when completing the tool in the selected patients' homes and did not categorize them because they did not know the cut off points of the tool (single blind assessment).

2. The first application of the tool was conducted by two examiners independently and separately, preferably during the same visit; after completing the tool, each examiner placed their sheet in an envelope without commenting on or seeing what the other examiner did.

3. The second application was conducted only by Examiner 1, with an interval of between four and seven days from the first application, because it was empirically observed by the team that the evolution of patient status does not initially change enough to alter their tool-generated category.

4. Sample randomization and masking of tool application were incorporated into the study design with the goal of eliminating external variations and minimizing sampling bias.

Supervision was performed remotely, with an agreement for weekly email exchange with the researcher. This communication displayed the number of visits that were scheduled, and completed or not completed because of refusals; discharge; death; hospital admission; change of address; and patient or residence not found after three scheduling attempts. In these cases, it was expected that the original patients would be replaced by other patients with the same profile until reaching a situation of "loss," that is, when there would be no patients left in the area with the same profile in the existing databases.

Excel spreadsheets were employed for data processing; printed tools were typed and checked after delivery.

Validity and reliability

Sensitivity and specificity measures were considered when estimating validity; the resulting tool with scores was used for testing and the gold standard was the categorization of HC types performed by the specialized health team (HC1: 13 to 24 points; HC2: 25 to 30 points; HC3: 31 to 36 points).

Sensitivity was defined as the percentage of patients categorized as HC2/HC3 (the target demographic of HC) by the tool score and by the reference test, that is, the gold standard (true positive). Specificity was defined as the percentage of patients not categorized as HC2/HC3 by the tool score and by the reference test, that is, the gold standard (true negative).

For each tool score, researchers measured sensitivity and specificity in order to draw a receiver operating characteristic (ROC) curve with the respective 95% confidence intervals. This was done to find the optimal cut-off point.

The ROC curve is a type of graphic that estimates the best balance between sensitivity and specificity, based on the cut-off points of the instrument assessed. The curve makes it possible to evaluate the discriminatory power of the test. For tests with good discriminatory power, as sensitivity increases, specificity slightly decreases or remains unchanged until high levels of sensitivity are reached ⁽²⁴.

In order to employ the tool in service practice, researchers opted to take into consideration only two categories: HC1 and HC2/HC3, because most of the demand is centered on the two profiles that outline the specific action of primary care teams and home care teams, respectively.

When studying tool reliability, the Kappa statistic was employed to measure result agreement among examiners and among the same examiner's results, observing the proportions obtained in the first and second applications of the instrument. Researchers opted for the weighted Kappa calculation. The chosen weight system was square weighted deviation. For interpretation, the Landis and Koch agreement scale was chosen²⁵.

The software SAS 9.3 was used for statistical analyses, with a significance level of 5%.

The research project was approved the Research Ethics Committee of the Foundation for Learning and Research in Health Sciences of the State Health Secretariat of the Federal District, Project No. 392/2008. All examined patients and/or caregivers and family members signed a free and informed consent form.

Results

In the first stage of study (2012), the tool was applied to the 96 patients in the random sample; in the second stage (2013), it was applied to 157 patients, with a total of 253 patients. Of these, 10 patients were excluded during the validity study (3%); 29 patients were excluded during the reliability assessment for the same examiner's results at two different times (11.4%); and 13 patients were excluded during reliability assessment among different examiners at the same time (5%).

The tool's validity was supported by Examiner 1's score in the first assessment, in which it was found that of the 123 patients empirically classified as HC1, 36 (29%) scored less than 13 points and, of the 33 patients empirically classified as HC3, three scored above 36 (10%), both in relation to the established gold standard.

The ROC curve was constructed based on measurements of sensitivity and specificity calculated for each score given by the tool (Figure 1), with the following cut-off points (the area under the curve was equal to 0.694 with CI [95%] limits between 0.627 and 0.760):

Figure 1
ROC Curve with some cut-off points marked-Brasília, Federal District, Brazil, 2013.

The sensitivity and specificity of the tool were analyzed by considering different cut-off points. The best balance, as provided by the ROC curve, between these two classifications and the gold standard at a cut-off score of 21 (Table 2).

Table 2
Best cut-off point with gold standard defined by the ROC Curve - Brasília, Federal District, Brazil, 2013.

Care type	Gold standard
	Sensitivity (IC 95%)	Specificity (IC 95%)
HC2/HC3 (>21)	75.56	24.44
HC1 (≤21)	46.73	53.27

The following results were found in the reliability study (Table 3).

Table 3
Weighted Kappa test for assessment of the tool's reliability - Brasília, Federal District, Brazil, 2013.

Agreement	KAPPA index	Variation (IC 95%)
Same examiner	0.81	0.74-0.88
Different examiners	0.52	0.43-0.61

Discussion

The results of this study showed that the tool developed to assess patient complexity in HC presented evidence of both validity and reliability.

Thus, the tool may help in service organization, either on patient admission or when changing care type and in supporting the creation of therapeutic plans, contributing to define the frequency of visitations from health team professionals, transport logistics, and necessary appointments.

The authors observed that 29% of patients assessed as HC1 scored below 13 points, which is the minimum score as defined by the gold standard. This fact may indicate patients who do not have the due care complexity to be monitored at home. A relevant portion of patients undergoing oxygen therapy who use the equipment only in the evening or for some hours during the day may be included in this profile.

There were also scores that exceeded the maximum limit of the HC3 category (10%), which suggested above average care complexity, represented by patients with sequelae and/or more severe comorbidities who require more frequent and longer care, especially in the presence of multiple pressure ulcers of various degrees, palliative oncological care, and difficulties with caregivers/families, among other factors.

In relation to the validity study, the identification of the best cut-off point for the tool-21-took into consideration the maximization of sensitivity for HC2/HC3, because that is the target demographic of HC. Its sensitivity was 75.5%, with CI [95%] limits at 68.3% and 82.8% and specificity of 53.2%, with CI [95%] limits at 43.8% and 62.7%), which can be considered an adequate set of parameters for the objective of this evaluation²³.

This assessment considered that it is better for the home care service to care for more patients in the HC2/HC3 profile and fewer patients in the HC1 profile. Caring for more of the HC2/HC3 profiles patients would result in absorbing the most severe cases-who would truly require attention from a multi-professional home care team, than those who would not present this care complexity and who could be absorbed by primary health teams.

Therefore, cut-off point 21 was chosen to differentiate between HC1 patients, who reached that score, and HC2/HC3 patients, who scored above 21.

The main limitation to the use of these estimates for assessing validity through sensitivity and specificity is the existence of a "gold standard" to which test results are compared, with the possibility of the true status of the patient being, if the information is available, a collection of exams considered more adequate, or other types of diagnoses that serve as references, as was the case with this research.

Despite this limitation, the literature includes many validity studies for scores based on gold standards that use, for example, clinical descriptions for early detection of pneumonia caused by Mycoplasma Pneumoniae in order to initiate antibiotic therapy²⁶, or tools that help parents to detect pain in their children in the postoperative period at home so that they can decide whether analgesic medication should be administered²⁷and even investigation of the contacts of individuals exposed to tuberculosis to validate a predictive score to develop active tuberculosis²⁸, among others.

Reliability analysis had satisfactory results for same-examiner agreement (Kappa coefficient 0.81), which is a general trend already observed, because it is a comparison among the same examiner's scores, which were obtained by visiting the same patient with an interval of between four to seven days. For different examiner agreement, the authors estimated a Kappa coefficient of 0.52 (moderate), which may suggest differing interpretations among examiners when assessing the tool's items. This may indicate the need for more thorough training and more time to discuss and address doubts about the tool's application.

Conclusion

The HC complexity assessment tool has been shown to be a useful tool, suitable for the routines of multi-professional HC teams, because it enables validated, reliable, and self-explanatory categorization of care type for HC patients, which makes it possible to improve service organization and care quality. The authors believe that continuing training, monitoring, and assessment of the tool are crucial for its improvement in the practice of HC services.

References

• ¹
  Marcolin GCA, Montenário JVC, Borges CM, Souza AR, Barbosa ACS. Panorama da Atenção Domiciliar do Sistema Único de Saúde (SUS): correlatividade com os serviços de atenção primária. Teoria Soc [Internet]. 2014 [citado 2016 fev. 13];22(2): 254-75. Disponível em: http://www.fafich.ufmg.br/revistasociedade/index.php/rts/article/view/196/142
• ²
  Brasil. Ministério da Saúde. Portaria n. 963, de 27 de maio de 2013. Redefine a Atenção Domiciliar no âmbito do SUS [Internet]. Brasília; 2013 [citado 2013 jun. 03]. Disponível em: Disponível em: http://www.jusbrasil.com.br/diarios/54869258/dou-secao-1-28-05-2013-pg-30
  » http://www.jusbrasil.com.br/diarios/54869258/dou-secao-1-28-05-2013-pg-30
• ³
  Oliveira Neto AV, Dias MB. Atenção domiciliar no Sistema Único de Saúde (SUS): o que representou o Programa Melhor em Casa? Divulg Saúde Debate [Internet]. 2014 [citado 2016 fev. 13];(51):58-71. Disponível em: Disponível em: http://cebes.org.br/site/wp-content/uploads/2014/12/Divulgacao-51.pdf
  » http://cebes.org.br/site/wp-content/uploads/2014/12/Divulgacao-51.pdf
• ⁴
  Brito MJM, Andrade, AM, Caçador BS, Freitas LFC, Penna CMM. Atenção Domiciliar na estruturação da rede de atenção à saúde: trilhando os caminhos da integralidade. Esc Anna Nery. 2013;17(4):603-10.
• ⁵
  Pires MRGM, Duarte EC, Gottems LBD, Figueiredo NVF, Spagnol CA. Factors associated with home care: support for care management within the SUS. Rev Esc Enferm USP [Internet]. 2013 [cited 2016 Feb13];47(3):648-56. Available from: Available from: http://www.scielo.br/pdf/reeusp/v47n3/en_0080-6234-reeusp-47-3-00648.pdf
  » http://www.scielo.br/pdf/reeusp/v47n3/en_0080-6234-reeusp-47-3-00648.pdf
• ⁶
  Boudry JF1, Bünzli D, Rilliot J, Studer JP, Villard G. Managed care ou réseau de soins intégrés. Rev Med Suisse. 2010;6(264):1838-9.
• ⁷
  Instituto Brasileiro de Geografia e Estatística. Síntese de Indicadores Sociais: uma análise das condições de vida da população brasileira [Internet] Brasília; 2013 [citado 2014 maio 25]. Disponível em: Disponível em: http://biblioteca.ibge.gov.br/visualizacao/livros/liv66777.pdf
  » http://biblioteca.ibge.gov.br/visualizacao/livros/liv66777.pdf
• ⁸
  Silva KL, Sena RR, Seixas CT, Feuerwerker LCM, Merhy EE. Home care as change of the technical-assistance model. Rev Saúde Pública [Internet]. 2010 [cited 2012 Nov 20];44(1):166-76. Available from: http://www.scielo.br/pdf/rsp/v44n1/en_18.pdf
• ⁹
  Banco Mundial. Relatório n. 36601- BR. Governança no Sistema Único de Saúde (SUS) do Brasil: melhorando a qualidade do gasto público e gestão de recursos [Internet]. Washington; 2007 [citado 2013 nov.7]. Disponível em: Disponível em: http://siteresources.worldbank.org/brazilinporextn/resources/3817166-1185895645304/4044168-1186326902607/19governancasusport.pdf
  » http://siteresources.worldbank.org/brazilinporextn/resources/3817166-1185895645304/4044168-1186326902607/19governancasusport.pdf
• ¹⁰
  Vituri DW, Lima SM, Kuwabara CCT, Gil RB, Évora YDM. Dimensionamento de enfermagem hospitalar: modelo OPAS/OMS. Texto Contexto Enferm [Internet]. 2011 [citado 2016 fev. 13];20(3):547-56. Disponível em: Disponível em: http://www.scielo.br/pdf/tce/v20n3/17.pdf
  » http://www.scielo.br/pdf/tce/v20n3/17.pdf
• ¹¹
  Martin LGR, Gaidzinski, RR. Creating and validating an instrument to identify the workload at an oncology and hematology outpatient service. Einstein [Internet]. 2014 [cited 2016 Feb13];12(3):323-9. Available from: Available from: http://www.scielo.br/pdf/eins/v12n3/pt_1679-4508-eins-12-3-0323.pdf
  » http://www.scielo.br/pdf/eins/v12n3/pt_1679-4508-eins-12-3-0323.pdf
• ¹²
  Possari JF, Gaidizinski RR, Lima AFC, Fugulin FMT, Herdman TH. Use of the nursing intervention classification for identifying the workload of a nursing team in a surgical center. Rev Latino Am Enfermagem [Internet]. 2015 [cited 2016 Feb 13]; 23(5):781-8. Available from: Available from: http://www.scielo.br/pdf/rlae/v23n5/0104-1169-rlae-23-05-00781.pdf
  » http://www.scielo.br/pdf/rlae/v23n5/0104-1169-rlae-23-05-00781.pdf
• ¹³
  Lachance J, Douville F, Dallaire C, Padilha KG, Gallani MC. The use of the Nursing Activities Score in clinical settings: an integrative review. Rev Esc Enferm USP [Internet]. 2015 [cited 2016 Feb 13];49(n. spe):147-56. Available from: Available from: http://www.scielo.br/pdf/reeusp/v49nspe/1980-220X-reeusp-49-spe-0147.pdf
  » http://www.scielo.br/pdf/reeusp/v49nspe/1980-220X-reeusp-49-spe-0147.pdf
• ¹⁴
  Genet N, Boerma WG, Kringos DS, Bouman A, Francke AL, Fagerström C et al. Home care in Europe: a systematic literature review. BMC Health Serv Res. 2011; 11(1):207.
• ¹⁵
  Salvador-Carullaet L, Alvarez-Galvez M, Romero C, Gutièrrez-Colosía M, Weber G, McDaid D et al. Evaluation of an integrated system for classification, assessment and comparison of services for long-term care in Europe: thee DESDE-LTC study. BMC Health Serv Res [Internet] 2013 [cited 2016 Feb 13];13(218):2-12. Available from: Available from: http://www.biomedcentral.com/1472-6963/13/218
  » http://www.biomedcentral.com/1472-6963/13/218
• ¹⁶
  Keeling DI. Homecare user needs from the perspective of the patient and carers: a review. Smart Homecare Technol Telehealth. 2014;2:63-76.
• ¹⁷
  Macdonald MT, Lang A, Storch J, Stevenson L, Barber T, Iaboni K et al. Examining markers of safety in homecare using the international classification for patient safety. BMC Health Serv Res [Internet]. 2013 [cited 2016 Feb 13];13:191. Available from: Available from: http://www.biomedcentral.com/1472-6963/13/191
  » http://www.biomedcentral.com/1472-6963/13/191
• ¹⁸
  Onderet G, Carpenter I, Finne-Soveri H, Gindin J, Frigters D, Henrard C et al. Assessment of nursing home residents in Europe: the Services and Health for Elderly in Long TERm care (SHELTER) study. BMC Health Serv Res [Internet]. 2012 [cited 2016 Feb 13];12:5. Available from: Available from: http://www.biomedcentral.com/1472-6963/12/5
  » http://www.biomedcentral.com/1472-6963/12/5
• ¹⁹
  Morris JN, Fries BE, Frijters D, Hirdes JP, Steel RK. interRAI home care quality indicators. BMC Geriatr [Internet]. 2013 [cited 2016 Apr 10];13:127. Available from: Available from: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3870973/
  » http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3870973/
• ²⁰
  Brasil. Ministério da Saúde; Secretaria de Atenção à Saúde, Departamento de Atenção Básica. Caderno de Atenção Domiciliar. Brasília: MS; 2013.
• ²¹
  Larson E, Cortazal M. Publication guidelines: need widespread adoption. J Clin Epidemiol. 2012;65(3):239-46.
• ²²
  Bôas MLCV, Shimizu H. Time spent by the multidisciplinary team in home care: subsidy for the sizing of staff. Acta Paul Enferm [Internet]. 2015 [cited 2015 June 13];28(1):32-40. Available from: Available from: http://www.scielo.br/pdf/ape/v28n1/en_1982-0194-ape-028-001-0032.pdf
  » http://www.scielo.br/pdf/ape/v28n1/en_1982-0194-ape-028-001-0032.pdf
• ²³
  Morini AJA, Carvalho GCM, Toshyiuki TM, Josiane F, Queiroz CLT, Fonseca VCF et al. Nursing Activities Score e carga de trabalho em unidade de terapia intensiva de hospital universitário. Rev Bras Ter Intensiva [Internet]. 2014 [citado 2016 abr. 10];26(3):292-8. Disponível em: http://www.scielo.br/pdf/rbti/v26n3/en_0103-507X-rbti-26-03-0292.pdf
• ²⁴
  Guessous I, Durieux-Paillard S. Validation des scores cliniques: notions théoriques et pratiques de base. Rev Med Suisse [Internet]. 2010 [cited 2015 June 13];6(264):1798-802. Available from: Available from: http://www.revmed.ch/rms/2010/RMS-264/Validation-des-scores-cliniques-notions-theoriques-et-pratiques-de-base
  » http://www.revmed.ch/rms/2010/RMS-264/Validation-des-scores-cliniques-notions-theoriques-et-pratiques-de-base
• ²⁵
  Szklo M, Javier Nieto F. Epidemiology: beyond the basics. 3rd ed. Burlington: Jones & Barlett Learning; 2014.
• ²⁶
  Ita JR, Torres-Quintanilla A, Paláu-Dávila L, Silva-Gburek JC, Elguea-Lizarrag JO et al. Score clínico para el descarte de neumonía por Mycoplasma pneumoniae. An Pediatr (Barc). 2014;81(4):241-5.
• ²⁷
  Ullán AM, Perelló M, Jerez C, Gómez E, Planas MJ, Serrallonga N. Validación de la versión española de la escala de evaluación del dolor postoperatorio Parent's Postoperative Pain Managament. An Pediatr (Barc). 2016;84(2):106-13.
• ²⁸
  Chan PC, Shinn-Forng Peng S, Chiou MY, Ling DL, Chang LY, Wang KF et al. Risk for tuberculosis in child contacts: development and validation of a predictive score. Am J Respir Crit Care Med. 2014;189(2):203-13.

Publication Dates

• Publication in this collection
  May-Jun 2016

History

• Received
  28 July 2015
• Accepted
  05 Apr 2016