Health technology assessment in Brazil: what do healthcare system players think about it?

Avaliação de tecnologias em saúde no Brasil: o que os atores do sistema de saúde pensam a respeito?

ORIGINAL ARTICLE

Health technology assessment in Brazil: What do healthcare system players think about it?

Avaliação de tecnologias em saúde no Brasil: O que os atores do sistema de saúde pensam a respeito?

Marcos Bosi Ferraz^I; Patricia Coelho de Soárez^II; Paola Zucchi^III

^IMD, PhD. Adjunct Professor, Department of Medicine, Universidade Federal de São Paulo - Escola Paulista de Medicina (Unifesp-EPM), São Paulo, Brazil

^IIDDS, MPH, PhD. Professor, Department of Preventive Medicine, Faculdade de Medicina da Universidade de São Paulo (FMUSP), São Paulo, Brazil

^IIIMD, PhD. Affiliated Professor, Universidade Federal de São Paulo - Escola Paulista de Medicina, São Paulo, Brazil

^{Address for correspondence}

ABSTRACT

CONTEXT AND OBJECTIVES: The health technology assessment (HTA) process has been developed locally. The aim of this study was to describe, analyze and compare the opinions of participants in international health economics symposia about the HTA process used in Brazil.

DESIGN AND SETTING: Observational cross-sectional study at the 2006 and 2008 International Health Economics Symposia, in São Paulo.

METHODS: A structured questionnaire was applied. For the statistical analysis, the percentage distribution for each category was calculated, and variables were compared using tests for two-sample proportion hypotheses.

RESULTS: Totals of 153 and 74 participants answered the 2006 and 2008 surveys, respectively. The response rate was better for the 2006 survey (67.1%) than for the 2008 survey (31.8%). Most interviewees were between the ages of 30 and 49 years and were managers in the healthcare system. Most of them considered that the current HTA process was incomplete and unable to meet the needs of the healthcare system. They mentioned the government, academia and experts as the three main groups of people who should be involved in the process, and selected efficiency/effectiveness, safety and disease relevance as the three main criteria to be considered in the HTA process. There is a trend towards developing decentralized regionalized HTA processes, with separate assessment and decision-making for the public and private systems.

CONCLUSIONS: The HTA concept is well known. Healthcare system players feel that the process has methodological limitations. Additional surveys are needed to track the HTA process and its application in Brazil.

Key words: Technology assessment, biomedical. Health facility administrators. Decision making, organizational. Health services administration. Health policy.

RESUMO

CONTEXTO E OBJETIVOS: O processo de avaliação de tecnologias em saúde (ATS) tem sido desenvolvido localmente. O objetivo deste estudo foi descrever, analisar e comparar as opiniões dos participantes dos simpósios internacionais de economia da saúde sobre o processo de ATS utilizado no Brasil.

TIPO DE ESTUDO E LOCAL: Estudo observacional transversal, realizado nos Simpósios Internacionais de Economia da Saúde de 2006 e 2008, em São Paulo.

MÉTODOS: Um questionário estruturado foi aplicado. A análise estatística calculou a distribuição percentual de cada categoria e comparou as variáveis com teste de hipóteses de proporções de duas amostras.

RESULTADOS: Um total de 153 e 74 participantes responderam às pesquisas de 2006 e 2008, respectivamente. A taxa de resposta foi melhor na pesquisa de 2006 (67,1%) do que na pesquisa de 2008 (31,8%). A maioria dos entrevistados estava entre 30 e 49 anos e era composta de gestores do sistema de saúde. A maioria considerou o atual processo de ATS incompleto e incapaz de satisfazer as necessidades do sistema de saúde. Entrevistados mencionaram o governo, academia e especialistas como os três principais grupos de pessoas que deveriam estar envolvidos no processo, e eficiência/efetividade, segurança e relevância da doença como os três principais critérios a serem considerados no processo de ATS. Existe a tendência de desenvolvimento de um processo de ATS descentralizado e regionalizado com avaliações e decisões separadas para o sistema público e privado.

CONCLUSÕES: O conceito ATS é conhecido. Os atores do sistema de saúde acham que o processo tem limitações metodológicas. Pesquisas adicionais são necessárias para acompanhar o processo de ATS e sua aplicação no Brasil.

Palavras-chave: Avaliação da tecnologia biomédica. Administradores de instituições de saúde. Tomada de decisões gerenciais. Administração de serviços de saúde. Política de saúde.

INTRODUCTION

The concept of health technology assessment (HTA) made its first international appearance in the 1960s, and has become an important decision-making tool for healthcare managers.¹ Since the late 1980s, HTA has grown exponentially in Europe, North America, Australia, Latin America and Asia.²

In Brazil, HTA started in 1983, with a seminar in Brasilia hosted jointly by the Pan-American Health Organization/World Health Organization (PAHO/WHO) and the Brazilian government. A number of aspects of HTA were discussed at this international event: political issues such as the questionable effectiveness of various healthcare technologies, issues of cost and cost-effectiveness and issues relating to technology transfer.³

Over the past six years, the HTA process has developed at an accelerated pace. In 2004, the Ministry of Health created its Department of Science and Technology (Departamento de Ciência e Tecnologia, DECIT) and approved the National Health Policy for Science, Technology and Innovation. In 2005, the General Coordination Office for Health Technology Assessment was created, with the mission of implementing, monitoring and disseminating HTA within Brazil's Unified Healthcare System (Sistema Único de Saúde, SUS). In 2006, procedures for incorporating technology into SUS were created. In 2007, the Brazilian Network for Health Technology Assessment (Rede Brasileira de Avaliação de Tecnologias em Saúde, REBRATS) was created to improve the government's regulatory capacity, and its ability to define prioritization criteria and methodology for HTA studies.^4-6 In 2009, the Institute for Health Technology Assessment (Instituto de Avaliação de Tecnologia em Saúde, IATS) was created to develop, foster and disseminate HTA in Brazil, thereby providing technical information to help in decision-making processes.⁷ The first national HTA Guidelines for conducting healthcare economic evaluations in Brazil were published in 2009.⁸ In 2011, Brazil is going to host the annual Health Technology Assessment International (HTAi) meeting, in Rio de Janeiro.⁹

Since 2000, the São Paulo Center for Health Economics (Centro Paulista de Economia da Saúde, CPES), at the Universidade Federal de São Paulo (Unifesp), has organized an annual health economics symposium in the state capital. Public and private healthcare professionals, managers and administrators involved in healthcare decision-making participate in the event. Over the years, this symposium has become a major forum of discussion for the main players in the Brazilian healthcare system, i.e. the Ministry of Health, state and municipal departments of health, service providers, healthcare insurers, regulatory agencies, the healthcare pharmaceutical and device industry, patient associations and academics with links to research and teaching institutions.

Every year, the theme for discussion is selected based on its relevance and on the need for a discussion on the topic, among the various healthcare players. The last three symposia (2006, 2007 and 2008) addressed HTA. Domestic and international participants contributed through descriptions of the challenges that they face in HTA processes; the extent to which HTA has helped (or hindered) the decision-making process; and/or the extent to which HTA has in fact improved the healthcare system in their countries.

Although HTA has been incorporated at the federal level, Brazil's regulatory framework is not yet complete. This means that HTA faces a major challenge with regard to its impact on how clinicians and healthcare system decision-makers think and act.^3,10,11 Regulating healthcare technology has a direct effect on different groups (patients, government, funders, the industry, etc.), and each of these may or may not be favored as a result of the process.¹² To initiate an awareness-raising process among these individuals, the first step is to understand their thoughts, feelings and expectations regarding HTA.

OBJECTIVES

In spite of the importance of this topic, little is known about the opinions of Brazilian healthcare players regarding the HTA process. To fill this gap, the aim of this study was to describe, analyze and compare the opinions of participants at the 2006 and 2008 International Health Economics Symposia in relation to the HTA process.

METHODS

Type of study, setting and participant sample

This was an observational cross-sectional study that used a structured questionnaire to investigate the opinions of a convenience sample from among the participants at the 2006 and 2008 International Health Economics Symposia.

Tool

The questionnaire used for this survey was split into two parts:

Part 1 - Participant description. A group of seven questions was used to describe the sample of professionals participating at the International Health Economics Symposium, with regard to the following: gender, age, professional activity within the healthcare system, type of system (public or private) and main segment of professional activity;

Part 2 - Health Technology Assessment. This comprised six questions to ascertain the participants' opinions on the process used to assess healthcare technology in Brazil: how it is done, who should participate, the criteria to be used and how the media and public opinion influence the process.

Procedure

The survey questionnaire was applied in a similar way during both the 2006 and the 2008 International Health Economics Symposia, by means of a one-hour interactive session that was included as part of the symposium program. The coordinator of the interactive session presented the purpose of the study and its intended use, and invited the audience to participate. Then, she explained in detail how to use the software (SunVote Professional Voting System, PVS-W52) to answer the questions. A few questions were used as examples to make the participants familiar with the use of the PVS-W52, which is an advanced two-way 2.4G radio frequency technology, with an easy-to-use keypad with 14 keys that is suitable for interactive sessions and audience response. It has multiple functions such as response feedback and number entering, which can be used to collect voters' polling. The coordinator read out each question, and both the question and the possible answerers were presented on a projection screen simultaneously. The interviewees were asked to give their personal opinion and not the opinion of the sector of activity that they represented. They were asked to press the keypad numbers corresponding to their answer, and had approximately one minute to accomplish this task. A number was assigned to each keypad, so that all the answers would be anonymous. The responses were sent to a collection center that automatically generated an Excel database, and the group results were immediately presented on a projection screen.

Data analysis

The participants' responses were analyzed using the Stata 8.0 statistical software. The categorical variables were described in terms of the number of individuals and percentage in each of the categories. Variables were compared using tests for two-sample proportion hypotheses. P values < 0.05 were considered significant.

RESULTS

Table 1 describes the International Health Economics Symposium participants who responded to the survey in 2006 (n = 153) and 2008 (n = 74). The response rate was better for the 2006 survey (61.7%) than for the 2008 survey (31.8%), with a statistically significant difference between the groups (P = 0.0000). The percentage of women in the sample fell from 55.6% to 41.3% (P = 0.0434). The percentage of participants in the age group between 30 and 49 years fell from 65.3% to 48.6% (P = 0.0181). Nonetheless, most of the participants in both events were between the ages of 30 and 49 years. There were no differences in terms of what type of healthcare system they worked in. In terms of where they performed their professional activities, the number of professionals linked to government or regulatory agencies increased considerably, from 24.0% to 42.6% (P = 0.0055), and the number linked to the healthcare pharmaceutical and device industry fell from 20.8% to 10.3% (P = 0.0011). Most of the respondents primarily held managerial or administrative positions. The number of participants working as providers fell from 13.5% to 8.8% (P = 0.0140), and the number of participants with links to research or teaching institutions increased (P = 0.0140).

The participants' opinions about the HTA processes are summarized in Table 2. At both symposia, the majority of the individuals (77.9% in 2006 and 86.8% in 2008) believed that the current HTA processes used in the public and private healthcare systems were methodologically incomplete, insufficient in scope and unable to serve the current needs of the healthcare system. Around 3.1% in 2006 and 7.4% in 2008 stated that the processes were incomplete and broad in scope and unable to serve the current needs of the healthcare system. Only 1.4% in 2006 and 5.9% in 2008 considered that the processes were complete, albeit insufficient in scope and unable to serve the current needs of the healthcare system.

With regard to who should be involved in the process of selecting the technologies that would be assessed, the responses were similar at the two symposia and mentioned the government and agencies, academic researchers and professional associations as the three main groups that should be involved, followed by the patients. In 2006, 10.8% of the respondents would include the lay public in the process used to prioritize technologies to be assessed, while in 2008, this number jumped to 23.4% (P = 0.0157). As expected, the groups consisting of consumer defense bodies and the healthcare pharmaceutical/device industry received the lowest scores (8.9%-10.9% and 17.8%-9.4%, respectively).

The interviewees selected evidence of effectiveness, relevance of the healthcare problem or impact on disease and evidence of risk and safety associated with use of the technology as the three main factors that should be used to prioritize which healthcare technologies to assess, followed by those relating to high cost of the technology and its impact on the budget (33.3%-46.2%) and lack of alternative for the disease or condition (22.8%-33.3%). Legal pressure and/or pressure from society received the lowest scores (3.0%-4.1%). The answers during the two symposia were similar.

With regard to the assessment process and decisions on whether to incorporate technologies into the public and private healthcare systems, opinions were divided during both symposia. The largest proportion of the 2006 participants (39.3%) felt that the assessment should be the same for both systems and that the decision to incorporate should be made by each system separately. Joint assessment and decisions to incorporate was the option selected by 32.1% of the participants, while 22.1% felt that it would be best if assessment and decisions to incorporate were handled separately by the public and private healthcare systems. In 2008, a large proportion of the participants (32.8%) also selected shared assessment and separate decisions to incorporate, but the same proportion of 32.8% selected separate assessment and incorporation decisions, while 26.6% felt that a single assessment and decision to incorporate should be used for both systems. These findings may suggest that the centralized single assessment process followed by a unified decision for the two systems is moving towards separate assessments and decisions to incorporate for the two systems.

Taking the regional differences in Brazil into consideration, most of the participants (45.0% and 43.3%, respectively) at both symposia suggested that the assessment process should be centralized but that the decision to incorporate healthcare technologies should be regional. In 2006, 37.1% of them chose a centralized unified assessment process and a national decision regarding incorporation, followed by 17.9% that preferred regional processes for both assessment and decisions to incorporate. In 2008, the scenario changed with a decrease in the percentage of those that preferred a centralized unified assessment process and an increase in the percentage of those that preferred regional processes for both assessment and incorporation decisions.

At both symposia, the vast majority of the decision-makers (71.6% in 2006 and 70.6% in 2008) felt that the media and public opinion currently favor incorporating new technologies in a manner that is both hasty and inadequate. Around 10% of them felt that the media and public opinion currently favor incorporating new technologies in a manner that is timely and adequate, and around 9% of them felt that the media and public opinion favored technological incorporation in a late and inadequate manner.

DISCUSSION

Health Economics Symposia are an important forum for creating awareness and teaching healthcare system players about the dilemmas and challenges of the HTA process. Above all, the forum has provided healthcare leaders and managers with an environment in which to discuss and reflect on the challenges that they face during the decision-making process. It has also promoted a dialogue among researchers and policymakers about how HTA can be a valuable tool for informing the decision-making process in Brazil.

In this study, we used volunteers who were available to us as our sample. The problem with this type of sample is that there is no evidence that such volunteers are representative of the population. Nevertheless, our main objective here was to describe this particular group in an exploratory way. Interviewing these 227 participants provided valuable insights into their opinions on the HTA process, even though this would not yield data on the proportion of healthcare players in the general population who might share these views. In general, the questionnaire responses on the HTA process were similar in 2006 and 2008. These two surveys may suggest rough estimates for the proportion of the population manifesting these opinions. When similar results are obtained repeatedly with different convenience samples, the likelihood that such results apply to the population is greater than when only a single convenience sample is used. Nevertheless, inferences based on such data must be made cautiously because of the possibility of hidden systematic bias.

Fewer participants provided responses in the 2008 (31.8%) survey than did in 2006 (61.7%). When people decline to participate in a study, they may do so for reasons that could be very important for the study results. This is more likely to happen when the study calls for revelation of deeply personal or embarrassing information, which was not the case here. The only difference that might have affected the response rate was the timing of questionnaire application. In 2006, the survey was applied on the first day of the symposium, just after the main lecture of the day. In 2008, the survey was applied in the afternoon of the last day of the symposium. Many participants come from outside São Paulo and often return home before the end of the event. The lower response rate may indicate that the 2008 sample consisted of self-selected participants, people who were motivated to participate, for some reason. Perhaps they were the ones more interested in the HTA process.

The differences do not invalidate the results, which were quite similar for the two symposia. We are confident about the responses provided and believe that this study captured the respondents' main opinions and made their position regarding the Brazilian HTA process clear.

Increased participation by individuals with links to the government (P = 0.0055) and with links to research and teaching institutions (P = 0.0140) may reflect increased attention to this topic among such individuals, and consequent willingness to participate in the survey as they came to recognize the importance of the topic. It may also be an indication of the importance that both of these groups placed on the HTA process in Brazil. The majority of the assessments conducted to date have been commissioned and funded by the government, and have been carried out by researchers affiliated to local universities.

Regarding the question that explored the HTA process currently used in Brazil, the majority of the participants in both symposia considered that the process was methodologically incomplete. Similar findings were presented in the paper by Pichon-Riviere et al., in relation to whether the 15 key principles for guiding decisions on structure, methods, processes and use of HTA are relevant and useful in Latin America. Decision-makers in 18 Latin American countries, including Brazil, considered that Principle 5 (HTA should incorporate appropriate methods for assessing cost and benefits) and Principle 7 (A full society perspective should be considered when undertaking HTAs) were very relevant, but they stated that intermediate to large-magnitude "gaps" existed, i.e. mismatches between the importance of HTAs and their real application. This was taken to mean that there was a high level of perceived non-application of these methodological features in HTAs in Latin America, which also perhaps reflected an acceptable level of development of HTA capacity in the region.¹³

The interviewees' opinion in the present study that the current HTA process was incomplete, insufficient in scope and unable to meet the needs of the healthcare system may be explained by the fact that the development of HTAs, although rapidly growing, is still embryonic in Brazil. Most of the assessments available through the website of DECIT, the Ministry of Health's body responsible for coordinating HTA activities in Brazil, are systematic reviews⁵ and not methodologically complete HTAs. After summarizing the evidence, other methodological steps are necessary to finalize the assessment and make suitable recommendations. An international study looking at the best way to incorporate effective technologies in 12 OECD (Organization for Economic Cooperation and Development) nations showed that only 14% of the HTAs could be categorized as broad assessments of medical technology.¹⁴ However, although neither complete nor broad, they seem to have presented decision-makers with the information that they need and to have met the goals of their healthcare systems.¹⁵ To illustrate with an example, the Ministry of Health's Brazilian Bulletin for Health Technology Assessment showed that one systematic review on a product of uncertain clinical usefulness gave rise to a reduction in the annual budget requirement of around 800 million reais in 2005.⁶

The pace at which new knowledge is acquired and the constant and growing offer of new technology, allied with the speed at which information now flows due to modern information and communication media, makes health technology assessment a true challenge to overcome. The need to develop local assessment processes and respect the characteristics of the healthcare system and resource availability pose an additional challenge. Not only is HTA a relatively new process within the international context, and one for which the methodology is still being developed, but also there is a need for qualified people to conduct and critically assess the process, results and recommendations so that HTA may guide the decisions made on behalf of the healthcare system. There is a shortage of technically qualified personnel to perform HTA studies at the national level, which has led to a perception that the process is not broad enough, given that it leaves out numerous technologies.

Thus, although the HTA process flow has existed since 2006, it is still seen as slow, unable to come to a resolution and unable to meet the needs of the healthcare system. The delays and the time required for assessments have also been the subject of criticism in developed nations such as Canada, the UK and Australia, where there are well-established HTA agencies and the process lasts around 11 to 18 months.^16-18 In general, HTA evaluators and users recognize that the time that it takes to make a complete scientifically sound HTA is paradoxical to the urgent need for information in order to make a decision.¹⁹ The so-called technical time is considered to be unrealistic and incompatible with the political decision-making time. However, it is important to stress that the quality and scope of the HTA process can be impacted by the volume of resources and time available.²⁰ The impact of HTAs depends on timely application in decision-making and subsequent implementation.²¹

In this study, the responses consistently pointed towards the government (Ministry and Departments of Health), researchers and professional associations as the three main groups that should be involved in the process for selecting and prioritizing technologies for assessment. However, in our 2008 survey, we noted that there was greater mention of including the lay public, thereby reflecting what has already happened in other developed nations. For example, international agencies have included all players in their assessment process. In the UK, the National Center for Health and Clinical Excellence (NICE) allows patient groups to participate in all steps involved in developing a recommendation, from designing the scope of the research to including comments in the reports that are issued.²² Public interest in both the process and the prioritization of HTAs within decision-making is clear, and a role for "the public" is widely promoted.²³ The Brazilian HTA policy will have to respond to the challenges of building public involvement into the bodies involved.

The ranking of the three most important elements for assessing healthcare technology reflected the type of information that decision-makers felt was necessary. Between 80.7 and 84.8% believed that effectiveness was an important factor, 77.3-85.5% believed that the relevance of the health problem was important and 52.4-63.6% felt that safety/risk elements were important. These were the same factors as listed in a survey of managers in charge of healthcare funding decisions in OECD countries. In that survey, 91% of the participants felt that safety was an important element, 86% believed that effectiveness was important and 81% felt that the relevance of the healthcare problem or disease burden was important.²⁰ It is interesting to note that, although economic issues have always been present in national debates about healthcare funding and sustainability, the high cost of technology and its potential impact on the budget were not selected as one of the three main HTA criteria.

Given the connection between HTA and the bodies that formulate health policy, the structure of the HTA process is molded by the type of healthcare system within which it takes place. In countries in which healthcare is primarily government funded (e.g. Sweden, Canada, Norway, UK and Spain), government agencies have been created for this purpose. On the other hand, in countries with private healthcare systems (such as the United States), HTA is handled by numerous individuals or groups linked to pension funds, healthcare funders or universities.^15,24

In Brazil, although the healthcare system is mixed (the entire population of 190 million persons has the right to access SUS, while 41 million have access to the private healthcare system), the formal HTA structure is fundamentally linked to the public healthcare system. However, the split in responses to question 4 may demonstrate that there remain questions regarding whether there should be a single assessment process for the two systems. From 2006 to 2008, in spite of an increase in the number of professionals linked to the government or regulatory agencies that would potentially in charge of creating a public assessment body, and a decrease in the percentage of professionals from the pharmaceutical and device industry, who might advocate different assessment processes for the public and supplemental healthcare systems, with less government interference, there was a slight increase in the number of responses selecting different assessments and decisions to incorporate, for the two systems.

This also points towards the potential for tension between, on the one hand, the pharmaceutical and device industry and the providers of private healthcare who wish to offer the newest and best products and services, and thus prefer an HTA process that is quick, and, on the other hand, the SUS public policymakers who, because of lack of funds, must ensure that technologies are incorporated rationally so as to ensure the sustainability of the public healthcare system over time.

CONCLUSION

The concept of HTA was well-known among leaders and decision-makers. Public and private healthcare system managers felt that the process had methodological limitations and did not fulfill the healthcare system needs. The leaders clearly revealed the main groups who should be involved and the most important criteria that should be applied to the HTA process. There was a trend towards developing decentralized regional HTAs, with different assessments and decisions to incorporate, for the public and private systems.

In spite of the challenges ahead for HTA, and moments of conflict among the various players, it is a tool that can contribute towards building an effective and efficient healthcare system. This study can help in focusing on factors that will make it possible to track the development of the HTA process and its implementation in Brazil.

Date of first submission: May 31, 2010

Last received: March 19, 2011

Accepted: March 22, 2011

Sources of funding: None

Conflict of interest: The authors are researchers at the Instituto para Avaliação de Tecnologia em Saúde/Conselho Nacional de Desenvolvimento Científico e Tecnológico (IATS/CNPq)

• 1. Banta HD, Luce BR. Health care technology and its assessment: an international perspective. New York: Oxford University Press; 1993.
• 2. Banta D, Jonsson E. History of HTA: Introduction. Int J Technol Assess Health Care. 2009;25 Suppl 1:1-6.
• 3. Banta D, Almeida RT. The development of health technology assessment in Brazil. Int J Technol Assess Health Care. 2009;25 Suppl 1:255-9.
• 4. Goldbaum M, Serruya SJ. O Ministério da Saúde e a política de ciência, tecnologia e inovação em saúde [The Brazilian Ministry of Health and science, technology, and innovation policy]. Cad Saude Publica = Rep Public Health. 2006;22(3):470-1.
• 5. Brasil. Ministério da Saúde. Secretaria de Ciência e Tecnologia. Insumos Estratégicos. Avaliação de Tecnologias em Saúde: institucionalização das ações no Ministério da Saúde [Health technology assessment: institutionalization of actions in the Brazilian Ministry of Health]. Rev Saude Publica = J Public Health. 2006;40(4):743-7.
• 6. Atallah AN. Evidence-based health technology assessments for the Brazilian National Health System (Sistema Único de Saúde, SUS) and for all. Sao Paulo Med J. 2009;127(2):59-60.
• ⁷
  IATS. National Institute of Science and Technology for Health Technology Assessment. Mission. Available from: http://www.iats.com.br/eng/missao.php?id_cms_menu=2 Accessed in 2011 (Mar 22).
• ⁸
  Brasil. Ministério da Saúde. Secretaria de Ciência, Tecnologia e Insumos Estratégicos. Departamento de Ciência e Tecnologia. Diretrizes metodológicas: estudos de avaliação econômica de tecnologias em saúde. Brasília: Ministério da Saúde; 2009.
• 9. Ministério da Saúde. Consolidação da área de avaliação de tecnologias em saúde no Brasil [Consolidation of health technology assessment in Brazil]. Rev Saude Publica = J Public Health. 2010;44(2):381-3.
• 10. Banta D. Health technology assessment in Latin America and the Caribbean. Int J Technol Assess Health care. 2009;25 Suppl 1:253-4.
• 11. Banta HD. Medical technology and developing countries: the case of Brazil. Int J Health Serv. 1986;16(3):363-73.
• 12. Lehoux P, Blume S. Technology assessment and the sociopolitics of health technologies. J Health Polit Policy Law. 2000;25(6):1083-120.
• 13. Pichon-Riviere A, Augustovski F, Rubinstein A, Martí SG, Sullivan SD, Drummond MF. Health technology assessment for resource allocation decisions: are key principles relevant for Latin America? Int J Technol Assess Health Care. 2010;26(4):421-7.
• 14. Poulsen PB, HØrder M. [Assessment of medical technology in practice]. Ugeskr Laeger. 1998;160(35):5041-4.
• 15. Neumann PJ. Lessons for health technology assessment: it is not only about the evidence. Value Health. 2009;12 Suppl 2:S45-8.
• 16. Drummond M, Sorenson C. Nasty or nice? A perspective on the use of health technology assessment in the United Kingdom. Value Health. 2009;12 Suppl 2:S8-13.
• 17. Hailey D. The history of health technology assessment in Australia. Int J Technol Assess Health Care. 2009;25 Suppl 1:61-7.
• 18. Menon D, Stafinski T. Health technology assessment in Canada: 20 years strong? Value Health. 2009;12 Suppl 2:S14-9.
• 19. Lehoux P, Hivon M, Denis JL, Tailliez S. Health Technology Assessment in the Canadian Health Policy Arena: Examining Relationships between Evaluators and Stakeholders. SAGE. 2008;14(3):295-321. Available from: http://evi.sagepub.com/cgi/content/abstract/14/3/295 Accessed in 2011 (Feb 18).
• ²⁰
  Organization for Economic Co-Operation and Development. OECD. Directorate for Science, Technologies and Industry. Health Technologies and Decision Making. Available from: http://www.oecd.org/document/55/ 0,3343,en_2649_34537_35589431_1_1_1_1,00.html Accessed in 2011 (Fev 18).
• 21. Sorenson C, Drummond M, Kristensen FB, Busse R. How can the impact of health technology assessments be enhanced? World Health Organization on behalf of the European Observatory on Health Systems and Policies; 2008. Available from: http://www.euro.who.int/__data/assets/pdf_file/0019/73225 /E93420.pdf Accessed in 2011 (Feb 18).
• 22. Drummond M, Banta D. Health technology assessment in the United Kingdom. Int J Technol Assess Health Care. 2009;25 Suppl 1:178-81.
• 23. Abelson J, Giacomini M, Lehoux P, Gauvin FP. Bringing 'the public' into health technology assessment and coverage policy decisions: from principles to practice. Health Policy. 2007;82(1):37-50.
• 24. Wild C, Gibis B. Evaluations of health interventions in social insurance-based countries: Germany, the Netherlands, and Austria. Health Policy. 2003;63(2):187-96.

Publication Dates

• Publication in this collection
  21 Sept 2011
• Date of issue
  2011

History

• Received
  31 May 2010
• Accepted
  22 Mar 2011
• Reviewed
  19 Mar 2011