Systematic review of theories: a tool to evaluate and analyze selected studies

Abstracts

O objetivo deste estudo é relatar a experiência de construção e utilização de um instrumento de captação e análise dos referenciais teórico-metodológicos de estudos, em revisões sistemáticas da literatura. O que se pretende é que investigadores disponham de um instrumento adequado para avaliar os estudos que expõem seus fundamentos teóricos, e que os profissionais de saúde tenham acesso a explicações teóricas para os resultados de estudos e suas aplicações nas práticas em saúde. Desarticulação entre teoria e prática pode levar à falta de motivação no local de trabalho e a práticas de reprodução de procedimentos sem consciência dos conceitos subjacentes que embasam a interpretação de um fenômeno saúde-doença. Chama-se a atenção dos pesquisadores no sentido de realizar análises sobre os fundamentos teóricos dos fenômenos saúde-doença em estudo e propõe-se questões relacionadas aos critérios de inclusão, apreciação crítica e extração de dados a serem abordadas em instrumentos.

Revisão; Medidas, métodos e teorias; Avaliação de Programas e Instrumentos de Pesquisa; Estudos de validação

REPORTS ON EXPERIENCE

Systematic review of theories: a tool to evaluate and analyze selected studies

Revisión sistemática de teorías: una herramienta para evaluación y análisis de trabajos seleccionados

Cassia Baldini Soares^I; Tatiana Yonekura^II

^INurse. Associate Professor of the Collective Nursing Department, School of Nursing, University of São Paulo. São Paulo, SP, Brazil. [e-mail]

^IINurse. Student of the Masters in Nursing Program, School of Nursing, University of São Paulo. Fellow of the State of São Paulo research Foundation. São Paulo, SP, Brazil. [e-mail]

ABSTRACT

The objective of this study is to report on the experience of constructing and using an instrument to collect and analyze theoretical-methodological references of studies, in systematic literature reviews. The goal is for researchers to have available an instrument that is appropriate for evaluating the studies that present their theoretical foundations, and for health professionals to have access to the theoretical explanations for study results and their applications in the practice of health care. The dissociation of theory from practice may lead to a lack of motivation at the work site and practices of repeating procedures without any awareness of the underlying concepts that base the interpretation of a health-disease phenomenon. Therefore we call on researchers to conduct reviews on the theoretical foundations of certain health-disease phenomena and we propose questions related to the inclusion criteria, critical appreciation and data extraction to be addressed in instruments.

DESCRIPTORS: Review; Measurements, methods and theories; Evaluation of Research Programs and Tools; Validation studies.

RESUMEN

Este trabajo apunta a relatar la experiencia de construcción y utilización de un instrumento de captación y análisis de referenciales teórico-metodológicos de estudios, en revisiones sistemáticas de literatura. Se pretende que los investigadores dispongan de un instrumento adecuado para evaluar los estudios que exponen sus fundamentos teóricos y que los profesionales de salud tengan acceso a aplicaciones teóricas para los resultados de estudios y sus aplicaciones en prácticas de salud. La desarticulación entre teoría y práctica puede llevar a falta de motivación en el lugar de trabajo, para prácticas de reproducción de procedimientos sin conciencia del concepto subyacente que da base a la interpretación del fenómeno salud-enfermedad. Se llama la atención de los investigadores en sentido de realizar análisis de fundamentos teóricos del fenómeno salud-enfermedad en estudio y se proponen cuestiones relacionadas al criterio de inclusión, apreciación crítica y extracción de datos a ser abordados en instrumentos.

DESCRIPTORES: Revisión; Mediciones, métodos y teorías; Evaluación de Programas e Instrumentos de Investigación; Estúdios de validación.

INTRODUCTION

Numerous studies in the healthcare field, particularly those from the countries in the center of capitalism, face strong obstacles in reaching healthcare providers in other countries in order for these studies to be immediately used. Access to knowledge follows the same standards of unequal access to tangible and intangible assets produced by human society.

Such acknowledgement, even if with different theoretical-practical nuances, is generalized. International authorities in charge of the world's social progress worry about this issue because, after all, it is related to the very development of mankind⁽¹⁾.

MEDLINE aptly illustrates this point. Produced by the U.S. National Library of Medicine, it is one of the main online international databases accessible in Latin American and the Caribbean and available through the Latin-American and the Caribbean Center of Information in Health Sciences, also known by its original name - Biblioteca Regional de Medicina (BIREME) - a specialized center of the OPAS/OMS oriented towards technical cooperation in scientific health information.

Consider the fact that the 4800 magazines indexed in the MEDLINE database are mainly publications from countries located in the northern hemisphere and from Australia, 3200 of which are published in English-speaking countries. Of those, 90% are published in North America or Western Europe (44% from the USA). The magazines of the so-called developing countries account for just 9% of the titles indexed⁽²⁾.

In the past several years it has been observed that there has been an increase in the scientific production of countries outside the United States/European community/Japan circuit, which seems to signal more investigative availability. However, this has not happened without a price⁽³⁾.

Although a significant part of the scientific production takes the format of academic merchandise⁽³⁾, here we are assuming that a part of the so-called scientific community⁽⁴⁾ creates literature reviews for the purpose of improving access to scientific production.

In the area of health they have become more and more frequent, constituting a potent tool to gather and analyze comprehensively and methodically the results of research studies coming from several continents. The purpose of a summary with the best findings is to establish recommendations to enable healthcare providers and the population in general to have access to knowledge that can be quickly and soundly used in and/or requested from healthcare services.

Comprehensive and time-unlimited literature reviews are currently called systematic reviews and are not exclusive to the health field. They are able to gather the best outcomes from the research available regarding a particular subject or topic. Usually the objective of a systematic review is to answer a question, and to do that it uses objective, clear and transparent methodological procedures to find, evaluate and summarize those research findings, using a pre-designed instrument to handle each one of those phases.

Thus, a number of research centers have been formed dedicated to stimulating and monitoring systematic literature reviews. Among the main centers, the following deserve mention: The Cochrane Collaboration, The Joanna Briggs Institute, The Campbell Collaboration, Centre for Reviews and Dissemination, EPPI Centre, NICE - National Institute for Health and Clinical Excellence, SCIE - Social Care Institute for Excellence, Critical Appraisal Skills Programme (CASP) and many others.

By no means are we suggesting taking systematic reviews and evidence-based medicine and/or practice - one of the more contemporarily stressed uses - as a practice deserving merit per se. Health is an attribute of life in society and does not equate with evidence; in addition, systematic reviews, even those conducted in centers whose strictness is acknowledged, can fail depending on the methodology used ⁽⁵⁾:

The instruments usually indicated for review-dedicated centers and used in the analyses of systematic reviews conducted by scientists in the health arena have proven to be adequate in capturing the results of qualitative and quantitative research projects in order to submit them to meta-analyses and/or meta-syntheses. The general objective of that type of work is to produce guidelines for actions in healthcare services.

The term meta-analysis expresses the analytical synthesis of research studies that use quantitative methods to capture the reality exactly because its objective is to combine statistically the gathered studies' results regarding a phenomenon of interest⁽⁶⁾.

The term meta-synthesis refers to the analytical analysis of a study's results using qualitative methods to capture the empirical objective. Its origin is in the education field, which seems to have initially used meta-ethnographic techniques to summarize huge sets of data in educational reports. It was followed by the sociology field, which diversified even more the objectives of the synthesizing process, including meta-theory and meta-method, among others. In the healthcare area, nursing is the profession that, drinking from that primary source, more often uses meta-synthesis⁽⁷⁾.

We have to highlight that integrative reviews, which have been part of the body of nursing investigations, constitute a method of systematic review among the traditional narrative reviews. Thus, they have the quality of being quite comprehensive because they can encompass quantitative and qualitative studies, analysis of theories and methods and even empirical research results, and so allow a more thorough approach to the phenomenon of interest. Their objective is to provide completeness to the theme because it is interesting to encompass virtually everything that has been studied about a subject. Once they enable reaching more complex objectives, with more amplitude, the synthesis obtained is able to contribute to overcoming problems related to healthcare⁽⁸⁾.

It is publicly known, at least by investigators, that much earlier and beyond this concern, historically there have been other interests when a literature review is conducted, the most common and the oldest being, in any area of science , the goal of learning state-of-the-art facts about a certain subject. It is the state-of-the-art fact about the phenomenon of interest that shapes the research's question and, in turn, defines more precisely the objectives of the studies⁽⁹⁾. Thus, it is an essential phase of every research study either to learn the results of previously conducted studies on the subject under consideration or to learn the theoretical references used when the subject studied is based on one or more approaches⁽¹⁰⁾. Similarly, a review offers the possibility of advancing the elaboration of theories and also guiding political and programmatic decisions⁽¹¹⁾.

Traditionally, scientists conduct what has become known as a narrative review of the literature. Although there are criticisms regarding the subjective nature of this type of review, investigators in the area of human behavior, for instance, draw attention to the importance of narrative reviews; they have a different nature from that of a systematic review and one has to decide on one or the other by judging the potential of each one to meet the objectives proposed by the research⁽¹¹⁾.

According to our experience, narrative reviews have allowed advancements in theorizing on several healthcare practices, such as educational practices related to drugs and damage control and educational practices in healthcare provided by nurses. However, this type of review does not meet the excellence criteria determined by review centers. Therefore, it is important to add to the scope of systematic reviews, in the form of practical instruments, the capacity of discussing on the theoretical aspects that ground research, which are best presented by narrative reviews.

The objectives of this study are to discuss the importance of including theories and theoretical models in the scope of systematic reviews and to propose instruments able to capture theoretical and methodological references of studies when a researcher is performing a systematic review of a subject.

The purpose of this study is that investigators have an adequate instrument to collect and evaluate studies proposing theoretical foundations in a systematic review and, finally, that healthcare providers have access to theoretical explanations for studies' results when preparing manuals, protocols and other methods of guidance in healthcare services.

Inclusion of theories and theoretical models in systematic reviews: theoretical foundations

The epistemological debate on scientific investigation has ethical-political and theoretical-methodological dimensions. At the ethical-political level we can say that there is a certain perversion in the university's role as a privileged center of research. It is more and more leaving behind the role of being a center of criticism and creativity targeted towards the general common good to becoming an institution that meets private interests.

The university - and the research done therein - has moved towards meeting the market's needs, leaving aside its role of a social institution at the service of the public interests that finance it, stimulating an individualistic stance as to research and academic career, dissociating them progressively from an humanizing project purposed to articulate science with a social transformation project⁽⁹⁾.

According to the dictionary of philosophy, the scientific definition of theory is as follows:

3rd: By opposition to vulgar knowledge: what constitutes the object of a methodical conception, systematically organized and dependent, consequently, as to its format, from certain scientific decisions and conventions that do not belong to common sense⁽¹²⁾.

This third entry can be complemented with the fifth, which expresses a more general sense of theory:

5th: By opposition to science's detail: a broad synthesis that proposes to explain a large number of facts considered, as hypothesis, probable by most scientists at a certain time: the atomic theory; the cellular theory. The time of personal doctrines and systems has passed and, little by little, they have been replaced by theories that represent the current status of the science, and they give to this point of view the result of everyone's efforts. Cl. Bernard, Introd, à l'et. De la méd. exp, III pare, cap.IV, §4⁽¹²⁾.

It is also worthy for us to understand the meaning of the theory within the ambit of scientific knowledge as denoted by the following the entries:

A theory is a hypothesis verified after being submitted to the control of the mind and experimental criticism... But for a theory to remain valid it should always modify itself according to scientific progresses and be constantly submitted to verification and criticism according to the new facts coming up. If we considered a theory perfect and refrained from verifying it through scientific experience, it would become a doctrine (Cl. Bernard, ibid., p. 325)⁽¹²⁾.

The scientific community and, in a way, the whole society in general acknowledge that the support of theories used in scientific investigations is influenced by values of a non-epistemic nature. From the philosophy field come concerns regarding how those values can alter the types of data scientists collect from a certain area of knowledge, sometimes transforming apparently irrelevant results into crucial evidence. Consequently, non-epistemic values associated with a scientific finding may influence the evaluation of theories, which attributes to scientists the task of reflecting on how to deal with those values. Large finances for investigations come from private organizations interested in a research's results. Pharmaceutical companies, for instance, in 2002 pledged to research more than the entire budget of the National Institute of Health (NIH), i.e., US $24 billion in just that year. Thus, increasing efforts are required to reflect on both ethical aspects and epistemic unraveling arising from finances, which are more and more privatized, for scientific investigations⁽¹³⁾.

As can be observed, the ethical-political dimension is irrefutably interwoven with the theoretical-practical aspect. Thus, a researcher should, based on his ethical-political commitment to constructing knowledge, structure his research strictly upon theoretical-methodological bases on which science has been historically established. This requires deep knowledge of the theories composing the hard core of his area of study and relates it to the categories, concepts and particular notions of the object being explored in the study ⁽⁹⁾.

A systematic review of how often theories are used in the design of guidance manuals for clinical practices and in the definition of strategies to implement interventions showed that it is necessary that papers better clarify the use of the theory founding the intervention practices being implemented, and that researchers develop more clearly the logic of how the theory proposed operates in the study at hand. Texts regarding studies based on theories should express those theories clearly, including quoting the original literature of the theory being used. Additionally, there should be clarity regarding why and how a theory is adequate to explain the practice being implemented, thus justifying the intervention proposed⁽¹⁴⁾.

A nursing research study attempted to identify the theories, theoretical tables and conceptual models used in studies pertaining to stopping smoking. The authors emphasized the importance of stating the theory to inform how and why things work and how a variable can be related to another. We would like to stress that research results that are clearly based on a theory can produce impossible solutions when findings are disconnected from the theories formalized by science. The authors indicate that subsequent research on that subject should be better articulated with the theories. This is a fundamental condition for implementing policies in that area because formulators of policies need to access consistent syntheses in order to have healthcare practices implemented. The theories articulated with the findings tend to speak on behalf of findings consistent with the phenomenon studied and, in this way, can be understood and interpreted by the managers of the healthcare policies ⁽¹⁵⁾.

A review on the use of theories in an important Latin American public health magazine revealed that references to theories are infrequent in publications; just a few papers mention a theory or a theoretical model on which authors base their research to explain the phenomenon under investigation⁽¹⁶⁾.

Implications in healthcare practices

Many observations have been made about the increasing disarticulation between theory and practice in the qualification of healthcare providers and in healthcare services. Studies we have conducted evidenced that at least in the qualification of nurses, said disarticulation can be found when healthcare⁽¹⁷⁾ and collective healthcare⁽¹⁸⁾ education is provided where, similar to what happens in other areas, the teaching of procedures takes precedence over explaining the theory embedded in those procedures and the theory, often enough, simply disappears.

The unilateral approach present in conceptions that excessively emphasize practice and leave aside the theory is stressed here ⁽¹⁹⁾:

...human practice, included here the production of knowledge, always encompasses a relationship between the unique and private and the universal, which is a historical phenomenon once the subjective and objective human properties involved therein result from broad and complex relations between man and nature. When man transforms nature, he is transformed and develops skills, creates needs, which makes significantly complex his vital activity, i.e., he constitutes himself as a praxical being. It is in the articulating unit between idea and action, or between theory and practice that human historicity takes place, concretized in the movement of constitution of the social reality ⁽¹⁹⁾.

The authors endorse that for praxis to have meaning and be creative instead of just repetitive, theory and practice have to be articulated.

Using as image the plunging required in practice, we believe that an individual immersed in the immediate reality, without the support of concepts summarizing the historical experience of the human beings, is at risk of drowning in a multitude of chaotic information or, in the best scenario, to make slow and insignificant advances produced by unproductive splashing, like the one who has not learned how to swim and has been thrown into the water⁽¹⁹⁾.

Disarticulation between theory and practice may lead to lack of motivation in the work place, to the reproduction of procedures with a lack of awareness of the underlying concepts founding the interpretation of a health-disease phenomenon, and consequently to frustration because no solutions are produced with those practices, which would only be possible through idea-action and action-idea reflections.

The process of production in healthcare is part of the service area, the third sector in capitalist production. Like the other production processes, it uses Fordist/Taylorist, and more recently, Toyotist methods to organize work, which restates the theoretical-practical disarticulation and increasingly establishes the reiterative praxis. A recent study with nurses who work in primary healthcare units promotes the assistance model where the axle guiding the work is the planning of results, oriented towards rationalization instead of towards reflecting on the working process, which significantly wears out workers and produces endless repetitions of activities without allowing time and space to enable problem solving and advancements in the working processes⁽²⁰⁾.

That finding, among other classic ones arising from the analysis of the irremediable work division in the process of healthcare production⁽²¹⁾, also hegemonic in the public area⁽²²⁾, leads to huge dissatisfaction in the healthcare work in nursing. Thus, here we advocate that it is possible to overcome this alienation by qualifying and improving subjects so that they dominate the object, purpose and instruments in their work and by having them organize the work in ways that share the knowledge required for a creative praxis.

For Gramsci, we cannot prepare individuals for the more and more complex modern activities - with which science is so intimately interwoven - without holding as basis a general formative theoretical-practical and political culture. This preparation can qualify professionals able to reach the understanding about their insertion and their work within the society, able to work with technical knowledge, to incorporate the social issues and assume ethical stances capable to meet the specific needs of the social reality where they will perform⁽²²⁾.

This is not possible without theoretical-practical articulation!

Instruments proposed

Due to the proposal discussed above, which draws the attention of researchers in the sense of conducting analyses on the theoretical foundations of the health-disease phenomenon being studied, we propose that reviews be conducted which will be able to evidence the theories used in investigations, showing their articulation with the methodology and operational aspects. That assessment will certainly enable the feedback required so that values related to the necessary theoretical-practical articulation concretize new ways of production and/or divulging of knowledge, clearly stating their theoretical contents and their appropriateness to the characterization of the object being studied.

Therefore, and considering as a starting point the systematic review instruments made available by the JBI, we propose the following instruments related to initial assessment of the inclusion criteria targeting selecting studies within the ambit of review which describe the theory or theoretical model (Figure 1), critical assessment to evaluate the study and check methodological criteria (Figure 2), and data extraction aiming at describing the study's characteristics to ease data analysis (Figure 3).

Based on the initial assessment instrument (Figure 1), which checks if the study meets the inclusion criteria or not, it is possible to verify if a theory or theoretical model was described in the study. According to our experience so far, we attempted to certify that the research would at least refer to the theoretical frameworks already used. As for the instrument that performs a critical analysis of the selected studies (Figure 2), it has helped us to assess the components (categories, concepts and notions) of the theories used to sound the studies, as well as to analyze the extent to which the addressed theory is critically evaluated in terms of the possible empirical outcomes of the studies. Therefore, the strengths and weaknesses of the theories are observed in order to explain the results. The data collection instrument (Figure 3) aims to extract from the text any evidence of the analysis we performed on the theoretical components of the reviewed studies.

CONCLUSION

A work instrument should not be so fundamental that it could ease closed processes where the intentionality of a creative work disappeared and prevented more productive results from being achieved based on an organization of the work as a serial production line, as seems to happen with academic papers - where expressions such as high productivity and/or consumption of articles witness this trend. The academic work, as much as any other social praxis, should not and cannot allow this degree of alienation if it makes worker scientists mere reproducers of investigative procedures.

An investigator is an active worker, politically placed related to the object being studied, able to make choices related to theoretical references and methodological procedures capable of expressing the object better.

However, access to the technologies available in the society used in the proper place, i.e. as a means to reach purposes previously intended by the worker, is an important element in the construction of knowledge.

Therefore, we believe that in a systematic review the instruments should easily identify, analyze, describe and systematize data and, to a certain point, indicate a trend of what is being valued by the scientific community. If the instruments available included a proper epistemological dimension of the academic work, investigators would be helped in their work of constructing knowledge.

Our experience using the instruments proposed herein has shown several potentialities: by verifying, analyzing, and collecting the theoretical elements of the studies, it was possible to present to researchers and general health care workers, in a clear and systematic manner, the advancements and limitations of health care studies and practices that use the reviewed theories. This process promotes de full development of undergraduate and graduate students.

Scientists have to demand more and more from the institutions that conduct or register systematic reviews, the inclusion of theoretical reviews or reviews concerned with the theoretical dimension of the empirical work, either with a qualitative or quantitative nature, or both.

Hardly ever have we found instruments available for this type of review, prepared to handle designs of empirical research. Here we propose to continue and improve a trend set in this sense, already used in the JBI, which also shelters opinion studies and makes available instruments to perform them.

REFERENCES

• 1. Annan K. A challenge to the world's scientists [editorial]. Science [Internet]. 2003 [cited 2010 May 30];299(5612):1485. Available from: http://www.sciencemag.org/cgi/content/short/299/5612/1485
• 2. Kotzin S. Journal selection for Medline. In: 71th World Library and Information Congress: IFLA General Conference and Council "Libraries - A voyage of discovery"; 2005 Aug 4th - 18th; Oslo, Norway [Internet]. Oslo; 2005 [cited 2010 June 3]. Available from: http://www.ifla.org/IV/ifla71/Programme.htm
• 3. Castiel LD, Sanz-Valero J. Entre fetichismo e sobrevivência: o artigo científico é uma mercadoria acadêmica? Cad Saúde Pública. 2007;23(12):3041-50.
• 4. Kuhn T. A estrutura das revoluções científicas. 7ª ed. São Paulo: Perspectiva; 2003.
• 5. Castiel LD, Póvoa EC. Medicina Baseada em Evidências: "novo paradigma assistencial e pedagógico"? Interface Comun Saúde Educ. 2002;6(11):117-21.
• 6. Higgins JPT, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 updated March 2011 [Internet]. Melbourne: The Cochrane Collaboration; 2011 [cited 2011 June 10]. Available from: www.cochrane-handbook.org
• 7. Booth A. Cochrane or cock-eyed? How should we conduct systematic reviews of qualitative research? Paper presented at the Qualitative Evidence-Based Practice Conference, Taking a Critical Stance, Coventry University; 2001 May 14-16 [Internet]. [cited 2010 June 3]. Available from: http://www.leeds.ac.uk/educol/documents/00001724.htm
• 8. Mendes KDS, Silveira RCCP, Galvão CM. Revisão integrativa: método de pesquisa para a incorporação de evidências na saúde e na enfermagem. Texto Contexto Enferm. 2008;17(4):758-64.
• 9. Salum MJL, Queiroz VM, Soares CB. Pesquisa social em saúde: lições gerais de metodologia - a elaboração do plano de pesquisa como momento particular da trajetória teórico-metodológica. In: Anais do 2º Congresso Brasileiro de Ciências Sociais e Saúde; 1999; São Paulo, Brasil.
• 10. Alves-Mazzotti AJ. O método nas ciências naturais e sociais: pesquisa quantitativa e qualitativa. 2ª ed. São Paulo: Pioneira; 1999.
• 11. Guzzo RA, Jackson SE, Katzell RA. Meta-analysis analysis. Res Organ Behav. 1987;9(3):407-42.
• 12. Lalande A. Vocabulário técnico e crítico da filosofia. 3ª ed. São Paulo: Martins Fontes; 1999.
• 13. Elliott KC, McKaughan DJ. How values in scientific discovery and pursuit alter theory appraisal. Philos Sci. 2009;76(5):598-611.
• 14. Davies P, Walker AE, Grimshaw JM. A systematic review of the use of theory in the design of guideline dissemination and implementation strategies and interpretation of the results of rigorous evaluations. Implement Sci. 2010;5:14.
• 15. O'Connell KA. Theories used in nursing research on smoking cessation. Annu Rev Nurs Res. 2009;27(1):33-62.
• 16. Arana GAC. Use of theories and models on papers of a Latin-American journal in public health, 2000 to 2004. Rev Saúde Pública. 2007;41(6):963-9.
• 17. Almeida AH. Educação em saúde: análise do ensino na graduação em enfermagem no estado de São Paulo [tese doutorado]. São Paulo: Escola de Enfermagem da Universidade de São Paulo; 2009.
• 18. Campos CMS, Soares CB, Trapé CA, Buffette BR, Silva TC. Articulação teoria-prática e processo ensino-aprendizagem em uma disciplina de Enfermagem em Saúde Coletiva. Rev Esc Enferm USP. 2009;43(n.esp 2):1226-31.
• 19. Abrantes NA, Martins LM. A produção do conhecimento científico: relação sujeito-objeto e desenvolvimento do pensamento. Interface Comun Saúde Educ. 2007;11(22):315-25.
• 20. Santos VC, Soares CB, Campos CMS. A relação trabalho-saúde de enfermeiros do PSF no município de São Paulo. Rev Esc Enferm USP. 2007;41(n.esp):777-81.
• 21. Mendes Gonçalves RB. Tecnologia e organização social das práticas de saúde: características tecnológicas do processo de trabalho na Rede Estadual de Centros de Saúde de São Paulo. São Paulo: Hucitec; 1994.
• 22. Bujdoso YLV, Trapé CA, Pereira EG, Soares CB. A academia e a divisão social do trabalho na enfermagem no setor público: aprofundamento ou superação? Ciênc Saúde Coletiva. 2007;12(5):1363-74.

Correspondence addressed to:

Publication Dates

• Publication in this collection
  12 Jan 2012
• Date of issue
  Dec 2011

History

• Received
  23 June 2010
• Accepted
  11 Apr 2011