E-survey with researchers, members of ethics committees and sponsors of clinical research in Brazil: an emerging methodology for scientific research

Inquérito pela Internet com pesquisadores clínicos, membros de comitês de ética e patrocinadores de pesquisa clínica no Brasil: uma metodologia emergente para pesquisa científica

ORIGINAL ARTICLE

E-survey with researchers, members of ethics committees and sponsors of clinical research in Brazil: an emerging methodology for scientific research

Inquérito pela Internet com pesquisadores clínicos, membros de comitês de ética e patrocinadores de pesquisa clínica no Brasil: uma metodologia emergente para pesquisa científica

Sonia Mansoldo Dainesi; Moisés Goldbaum

Department of Preventive Medicine of the São Paulo University, School of Medicine - USP^*

^{Correspondence to}

ABSTRACT

INTRODUCTION: The growth of Internet users enables epidemiological studies to be conducted electronically, representing a promising methodology for data collection.

METHODS: Members of Ethics Committees, Clinical Researchers and Sponsors were interviewed using questionnaires sent over the Internet. Along with the questionnaire, participants received a message explaining the survey and also the informed consent. Returning the questionnaire meant the consent of the participant was given. No incentive was offered; two reminders were sent.

RESULTS: The response rate was 21% (124/599), 20% (58/290) and 45% (24/53) respectively for Ethics Committees, Researchers and Sponsors. The percentage of return before the two reminders was about 62%. Reasons for non-response: participant not found, refusal to participate, lack of experience in clinical research or in the therapeutic field. Characteristics of participants: 45% of Ethics Committee participants, 64% of Researchers and 63% of Sponsors were male; mean age (range), respectively: 47 (28-74), 53 (24-72) and 40 (29-65) years. Among Researchers and Sponsors, all respondents had at least a university degree and, in the Ethics Committees group, only two (1.7%) did not have one. Most of the questionnaires in all groups came from the Southeast Region of Brazil, probably reflecting the highest number of clinical trials and research professionals in this region.

CONCLUSION: Despite the potential limitations of a survey done through the Internet, this study led to a response rate similar to what has been observed with other models, efficiency in obtaining responses (speed and quality), convenience for respondents and low cost.

Keywords: Survey. Internet. Clinical research. Ethics. Questionnaire. CHERRIES.

RESUMO

INTRODUÇÃO: O crescimento de usuários da Internet possibilita que estudos epidemiológicos sejam conduzidos eletronicamente, representando uma promissora metodologia para coleta de dados.

MÉTODOS: Membros de Comitês de Ética, pesquisadores clínicos e patrocinadores foram entrevistados com questionários enviados pela Internet. Com o questionário, os participantes receberam mensagem explicando a pesquisa, além do termo de consentimento. O retorno do questionário significava que o consentimento havia sido dado. Nenhum incentivo foi oferecido; dois lembretes foram enviados.

RESULTADOS: A taxa de resposta foi de 21% (124/599), 20% (58/290) e 45% (24/53) respectivamente para os Comitês de Ética, pesquisadores e patrocinadores. A porcentagem de retorno antes dos lembretes foi de 62%. Razões de não resposta: participante não encontrado, recusa em participar, falta de experiência com pesquisa clínica ou com a área terapêutica. Características dos participantes: 45% dos membros de Comitês de Ética, 64% dos pesquisadores e 63% dos patrocinadores eram do sexo masculino; média de idade, respectivamente: 47 (28-74), 53 (24-72) e 40 (29-65) anos. Entre os pesquisadores e patrocinadores, todos os respondedores tinham curso universitário no mínimo e, no grupo de Comitês de Ética, somente dois (1,7%) não tinham. A maioria dos questionários em todos os grupos veio da região Sudeste do Brasil, refletindo provavelmente o maior número de pesquisas clínicas e profissionais de pesquisa nessa região.

CONCLUSÃO: Apesar das potenciais limitações de um levantamento conduzido pela Internet, esse estudo obteve taxa de resposta similar àquela observada com outros métodos, eficiência na obtenção das respostas (velocidade e qualidade), conveniência para os respondedores e baixo custo.

Palavras-chave: Inquérito. Internet. Pesquisa clínica. Ética. Questionário. CHERRIES.

Introduction

Projects involving interviews and questionnaires are usually carried out through the telephone, personal interviews or by mail (postal services), making use of paper-and-pencil questionnaires. Surveys over the internet seem to be an alternative and could be performed in two ways: sending the questionnaire by e-mail address or posting the questionnaire on the web (web-based survey, usually in HTML format).

The growth of the Internet in recent years in the world as well as in Brazil has been huge. Brazil is the 5th largest country in number of connections: there were around 67 million Internet users in December 2009, compared to 32.5 million users in 2006, 39 million in 2007 and 62.3 million in 2008. In urban areas, 44% of the population was connected to the Internet as well as 97% of enterprises. Thirty-eight percent of Brazilians accessed the Internet daily, 87% entered the Internet once a week, whether at work or at home. The entry of class C in this group of Internet users should boost growth in coming years^1,2. An update of this information in May 2010 showed 73 million Internet users in Brazil. Of these, 12% were between 6 and 14 years of age, and 56% between 15 and 34 years (2). Comparatively, in the United States, about 60% of homes were connected to the Internet in 2001 and 75% in 2004. In contrast, the number of fixed telephones in the world has been decreasing since 2007, although in Brazil this number is still relatively stable. On the other hand, the use of mobile phones increased 53.6% in the 2005-2008 quadrennial, in Brazil, compared to the previous quadrennial data (PNAD 2009); the largest leap, in proportional terms, occurred among low income groups².

An issue not fully answered is the appropriateness of using the Internet for scientific/academic research. This article brings the results of a survey done through the internet, in the area of clinical research, in that the main stakeholders in the field were the participants of the survey: investigators, members of ethics committees, sponsors and patients.

Methods

In this study, we worked with a convenience sample. Potential identified participants without an e-mail address were excluded from the sample.

Definition and selection of participants

Research stakeholders interviewed in this study were clinical investigators (RES) in the areas of diabetes mellitus (DM) and AIDS, members of Ethics Committees (EC); private sponsors (SPO), that is, pharmaceutical companies and Clinical Research Organizations (CRO) and patients (in the same therapeutic areas already mentioned).

The identification of each group of respondents was carried out as follows:

• EC: identification of contact members of ECs was based on a list of ECs accredited by CONEP (Comissão Nacional de Ética em Pesquisa) as of May 2009;

• Sponsors: private sponsors were defined as follows: (i) Pharmaceutical companies based in Brazil and doing research in the country; (ii) representatives of CROs in Brazil, identified through their association, called ABRACRO;

• Clinical researchers: the list of researchers was provided by the sponsors of clinical trials (pharmaceutical industries and/or CROs). Additionally, the

  www.clinicaltrials.gov webpage was explored looking for investigators working on studies conducted in Brazil in the pertaining therapeutic fields (DM and AIDS);

• Patients: The patients were included in the survey through their physicians who accepted to participate in the research. The patients answered the questionnaires in paper and the results are described elsewhere

  ³.

Development of questionnaires

Three questionnaires were used: one for investigators, one for sponsors and ECs and a third one for patients. Questionnaires were standardized as much as possible; however, given the different profiles of the participants, some changes had to be introduced. Questionnaires were based on those already used in the literature^4,5, as well as in interviews and meetings held with several experts in clinical research. Questions were modified aiming to remove those unrelated to the objectives of the current project and included additional topics, customizing the questionnaires to the therapeutic field and also to local language. Most questions were multiple choice. Four domains were explored:

• demographic characteristics of participants;

• experience and satisfaction with the completion of informed consent and the application process;

• reason/motivation for participating in clinical research; and

• opinions and views on the continuity of treatment after the study.

The survey questionnaires for this study were designed following the STROBE (Strengthening the Reporting of Observational Studies in Epidemiology)⁶, which are the recommended guidelines for reporting observational studies (www.strobe-statement.org), as well as the CHERRIES (Checklist for Reporting Results of Internet E-Surveys) ⁷, which refers to the reporting of Internet surveys.

Questionnaires were evaluated (pilot test) before being sent to participants: they were submitted to a panel of experts (two lawyers with experience in clinical research, two members of ECs, two researchers, one secretary and four representatives of sponsors). The method is commonly used and, for this study, it was very valuable because several adjustments were made in the questionnaires after the comments received during the pilot phase.

Method for sending questionnaires

Participants were invited to participate by e-mail, using mailing lists. A message was sent to all, explaining the reason-why for the research, presenting the questionnaire and the informed consent, and requesting consent for their participation. Returning the completed questionnaire already meant that the consent had been given. Participants were asked to return the questionnaire within three weeks.

Participation in the survey was voluntary and no material incentive was offered for completing the questionnaire. As recommended in the literature, an introductory letter was used as "incentive" and sent by the São Paulo University e-mail, with the name of the authors of the survey, as well as a CAPES page link to the project. Additionally, a reminder was sent, extending the deadline for answers for another 15 days and thanking those who had answered the questionnaire. Another reminder was made, but just for those who had not previously answered the questionnaire. Questionnaires were sent to participants between October 2009 and January 2010.

Ethical approval: The protocol and additional documents (informed consents, questionnaires and cover letters) were approved by the Ethics Committee of the São Paulo University, School of Medicine.

Results

Response rates

One concern regarding e-surveys is related to the response rate. The response rate, calculated as Questionnaires received (returned)/Questionnaires sent, was measured in the pilot phase, where the questionnaires were sent to a panel of experts: 10 of 12 answered (83%) within a deadline of 10 days.

The response rates of the participants, according to the groups they belonged to, are described in the Table 1.

After the first mailing of questionnaires, the percentage of responses was 77/124 (62.0% of total received) in the case of ethics committees, 35/58 (60.3%) for researchers and 16/24 (66.6%) for sponsors. Therefore, by joining the three groups, we found that 62% of the final answers were obtained before reminders were sent. The reasons for non-response are listed below: (i) participant or entity not found (the message by e-mail returned to sender), (ii) refusal to participate, (iii) lack of experience with clinical trials or with the therapeutic field selected for the study, (iv) questionnaires lost or not reaching their final destination.

Unfortunately it was not possible to separate each of these reasons because the invitations were usually sent to more than one email address, when available. Therefore, upon receiving a return showing, for example, that the address was not found, this does not necessarily mean that the participant had not been found, because he/she could have answered through the second e-mail available. This methodological error probably resulted in a lower response rate than could have been obtained if we could check who actually received the questionnaire and in fact did choose not to respond. Table 2 shows participants' characteristics.

Discussion

This study was conducted over the Internet, with questionnaires sent by e-mail, in the form of attachment. Another model of survey by the Internet is the use of questionnaires in HTML format. This has been increasingly used due to the possibility of utilizing a link to the questionnaire and because data goes directly to the database. This also allows respondents to remain anonymous and facilitates the data entry, since the responses are sent directly to the database.

The quality of the information collection instrument is essential for the results in any survey and the same is observed in e-surveys^7,8. Research conducted through the Internet represents a promising methodology for data collection due to the high number of responses and the convenience to collect them; it also saves time and money. The expansion of information technology, electronic media and access to equipment makes the web an appealing alternative to the traditional paper-and-pencil questionnaires sent by mail, or even to interviews conducted by phone, since the growing replacement of fixed phones by mobiles makes it difficult to find people by telephone. Additionally, mobile phones do not always enable finding a potential participant able to answer a questionnaire properly and smoothly.

Several reasons can be mentioned as the rationale for the enthusiasm for this kind of study:

• access to larger samples than usually obtained with conventional data collection methods, allowing greater scope, including geographical;

• efficiency in data collection in terms of quality (completed questionnaires), which could also be observed in our study as well as in terms of timely response. A survey carried out by Smith et al. in 2001, through the web, obtained responses in 7 days, on average, and 89% of respondents answered on the same day.

  ⁹ In our survey, about 60% of questionnaires were returned after the first mailing (in three weeks);

• greater degree of perceived anonymity;

• easier recruitment in possibly embarrassing situations;

• convenience for the interviewees, who can answer any time they find it more comfortable;

• lack of interviewer bias;

• lower total costs, although initial activities (start-ups) can have substantial costs

  ^9,10,11.

Certainly, some disadvantages or limitations can also be listed for electronic submission of questionnaires:

• the population of the Internet may not be representative of the general population, leading to a selection bias, even though the sample can be considered as representing a subgroup of the total population; in order to reduce the sampling error, it is recommended to include entire populations of certain groups to avoid/reduce error

  ^10,12. This precaution was taken in this study, so that the entire available universe was invited to participate in the survey;

• reduced external validity (results not generalizable);

• costs to respondents if they use Internet access via dial-up, less frequent nowadays;

• difficulty to calculate the precise response rate, since it is not always possible to know exactly how many participants received the questionnaire (unless we use the control of message receipt by the sender, not available in all web systems);

• low response rates due to incorrectness of e-mail addresses, concern of a virus or even due to a pattern of deleting unknown or unsolicited messages

  ^12,13.

It is worth looking at some articles comparing surveys made over the Internet with traditional paper questionnaires, sent by conventional mail, or done by phone or by personal interviews. Given the evolution of information technology worldwide, data from the initial years of this century show interesting differences when compared to more recent ones (past 2 to 5 years), as can be seen below. A study by Ritter et al. published in 2004 evaluated the characteristics of responses of patients recruited over the internet, randomly assigned to participate in a survey by mail or by the Internet. They observed that participation was as good as, if not better, among those who received the questionnaire via the Internet in relation to who received it by mail. Additionally, the responses did not differ significantly between groups, although the questionnaires sent by the Internet required fewer reminders to achieve similar response rates¹⁴.

Leece et al. published a study where a group of orthopedic surgeons were randomly divided to participate in a research through the internet or by conventional mail. A significantly lower response rate for questionnaires sent over the Internet in relation to conventional mail was observed¹². The different findings between these two studies may be explained, for example, by the characteristics of participants, but also by the different recruitment strategies used. Ritter recruited participants through the Internet, which probably are more likely to answer a questionnaire on the web than the general population. Similarly, a survey by conventional mail would reach higher response rates when potential respondents were contacted by mail and not by the internet¹⁵. In a study conducted in the United States between 2003 and 2006 and published in 2008, Rankin et al. found similar results on questionnaires sent online and by telephone (participants could choose the method of response) ¹⁶.

Kongsved et al. assessed the response rate and completeness of questionnaires returned in a randomized study of Internet versus paper questionnaires (with prepaid envelope for reply). The study was conducted in Denmark between 2004 and 2005. The response rate was 18% for the Internet group and 73% for the paper. Non-responders received a reminder with the option of completing another form of the questionnaire. After this reminder, in which participants could choose between the two ways of responding, response rates were 64% versus 76.5%, respectively, for internet and paper (p = 0.002). For those who did it through the Internet, 98% of the questionnaires arrived completely filled in, compared to 63% in the paper version (p < 0.001). In this population, the response rate was therefore better for the paper version, although the completeness was better for the internet version¹⁷.

Cook, Dickinson and Eccles published in 2009 an observational study assessing the response rate to questionnaires sent over the Internet, compared to fax or conventional mail. Three hundred and fifty studies conducted in several countries were included between 1996 and 2005. The average response rate was 57.5% and it was higher when a reminder was sent, although this occurred only in half of the studies analyzed. The authors underline that it is important to be cautious when analyzing surveys that do not show the response rates, since this may hide a possible non response bias¹⁸. Our study received, as already mentioned, 62% of total responses after the first mailing of questionnaires, somewhat between 25 and 30% of responses after the first reminder, and the remaining responses after the second reminder.

Malin and Barrowman compared the administration of questionnaires on quality of life in heart failure, when sent over the Internet, with paper questionnaires distributed and completed in person or by mail¹⁹. The authors pointed out the possible selection bias by requiring skills in information technology as well as an appropriate Internet access. More responses would be expected from a population with a higher educational level and younger. The study, conducted between 2006 and 2007, showed a mean age of 51 years, which is younger, in fact, than the average age of 72, characteristic of patients admitted to the same hospital with this disease. However, in relation to the educational level, 60% of those who participated had college degrees, compared with 52% who have the same educational level in the general population.

The average age in our survey was 47 years (members of ECs), 40 years (sponsors) and 53 years (researchers). It is not, therefore, a very young population, which could have been selected simply due to higher skills and ease with digital media. There may have been, however, some bias among members of EC, regarding education, since there was a high percentage of professionals with specialization and doctorates among the respondents, constituting, perhaps, a sample with a higher educational level and access to digital media.

Some studies conducted in our country should also be mentioned. Data from Mazzon (1983) mention that the return rate of questionnaires mailed in Brazil varies according to the source that is sending the survey, with educational institutions as those which get the highest rate of responses (15%). Incentives increased the rate of return, especially if financial (sending of a symbolic value to repay the effort and goodwill of the respondent), with the return rate reaching 33%. Most of the questionnaires were returned in the first two weeks, suggesting that a follow-up after the first week is recommended²⁰.

Hippolyte et al. presented a study in 1996, discussing the use of information technologies by teachers at FEA (São Paulo University, School of Economics and Administration). They found an average rate of return of paper questionnaires of 30% compared with 8.2% with questionnaires sent by e-mail. The deadline for return by e-mail was relatively short: 80% of questionnaires returned in 10 days, with the maximum return period being 18 days. The authors also highlight the importance of Information Technology infrastructure when using the Internet as a tool²¹.

In our study, using the São Paulo University e-mail, the inbox capacity had to be increased, otherwise it could have led to a lower response rate (missed information), if not identified early. The response rate obtained in our series was about 20% to 21% for researchers and members of ECs, and 45% for sponsors.

In 1997, Silva et al. published a study where a paper questionnaire was sent to professors at USP, with the option of filling it in paper or through a website. Sixty-four percent of the 102 teachers who received the questionnaire answered, 64.6% by paper questionnaire and 35.4% using the Internet. At that time, there was a clear preference for paper-and-pencil questionnaires²².

An extensive review on the topic was presented by Vasconcelos and Guedes during a Seminar on Management, at FEA-USP in 2007. They assessed the advantages and limitations of electronic questionnaires answered by the Internet, in the context of scientific research. They emphasized the expansion of the Internet - the estimated number of users connected on the Internet was in 2007, 32 million, representing an increase of 542% compared to 2000 - showing the potential of this tool for conduction of scientific research. A review of secondary data by the authors reinforced many of these findings. The conclusion pointed out that, as all research methodologies have limitations, the researcher must be familiar with these problems and take preventive actions to minimize them by properly choosing the methodology that best suits one's goals²³.

A study conducted in 2010 in ten countries of Western Europe, about reimbursement of medicines, using a questionnaire sent by e-mail to patient associations, presented a return rate of 31% (21 of 68 associations). The authors emphasized the difficulty to select patient organizations: although being careful to seek a broad spectrum of diseases (diabetes, asthma, cardiovascular disease, multiple sclerosis and Cushing's disease, this representing a rare disease), there are so many associations for each one of them, that they cannot state that the organizations selected were indeed representative of all diseases²⁴.

In summary, the following aspects are considered critical in conducting surveys through the Internet:

• the issue of representativeness of the sample coming from the Internet (selection bias);

• self-selection of participants (volunteer effect);

• external validity of the data collected and, obviously;

• the ethics of procedures

  ^13,25.

Regarding the latter issue, if privacy is not assured, since the email address comes automatically with the answer, at least confidentiality of the information should be assured^12,13,25. In recent search tools available for conducting surveys over the Internet (web-based surveys), such as SurveyMonkey (www.surveymonkey.com) or equestiona (www.equestiona.com), this problem disappears, since the data is automatically sent to the database, keeping responders anonymous.¹³ If authors are aware of these limitations, electronic surveys could occupy an important role in scientific research in coming years. The possibility of combined use of strategies, according to the profile of the study, should also be considered, taking into account that this combination can increase the error due to lack of uniformity of measurements.

In 2004, Gosling et al. already stated that samples over the Internet may even be more representative than through traditional methods. Paper questionnaires would remain useful in the future only to populations with limited experience or access to internet¹¹.

Conclusion

Despite limitations inherent to a survey carried out through the Internet, this study shows that e-surveys can be a reliable and useful tool, compared to the classical ways of doing academic research (by phone, mail or personal interviews). The observed trend in past years clearly demonstrates the potential of this new tool in all areas of knowledge.

Acknowledgements: Miriam Souza and Rogerio R. do Prado, for the support with the database construction and for the statistical analysis, respectively.

Recebido em: 17/08/11

Versão final apresentada em: 17/08/11

Aprovado em: 16/03/12

• ¹
  Estatísticas, dados e projeções atuais sobre a Internet no Brasil. Available at: www.tobeguarany.com/internet_no_brasil.php [Acessado em 11 de agosto de 2010]
• ²
  Teleco. Inteligência em telecomunicações. Available at: http://www.teleco.com.br [Acessado em 11 de agosto de 2010]
• 3. Dainesi SM, Goldbaum M. Post-trial access to study medication: a Brazilian e-survey with major stakeholders in clinical research. J Med Ethics 2012; 38(12): 757-762.
• 4. Emery S, Abrams DI, Cooper DA, Darbyshire JH, Lane HC, Lundgren JD, Neaton JD and the ESPRIT Study Group. The evaluation of subcutaneous Proleukin^® (interleukin-2) in a randomized international trial rationale, design and methods of ESPRIT. Control Clin Trials 2002; 23(2): 198-220.
• 5. Lacativa PGS, Szrajbman M, Silva DASM, Melazzi ACC, Gregorio LH, Russo LAT. Perfil de sujeitos de pesquisa clínica em um centro ambulatorial independente. Ciênc Saúde Coletiva 2008; 13(3): 611-20.
• 6. Von Elm E, Altman DG, Egger M, Pocock SJ, Gotzsche PC, Vandenbroucke JP for the STROBE initiative. Guidelines for reporting observational studies. J Clin Epidemiol 2008; 61: 344-9.
• 7. CHERRIES (Checklist for Reporting Results of Internet e-Surveys). Eysenbach G. J Med Internet Res 2004; 6(3): 34.
• 8. Rosen T, Olsen J. Invited commentary: the art of making questionnaires better. Am J Epidemiol 2006; 164: 1145-9.
• 9. Smith B, Smith TC, Gray GC, Ryan MAK for the Millenium Cohort Study Team. When Epidemiology meets the internet: Web-based surveys in the Millenium cohort study. Am J Epidemiol 2007; 166: 1345-54.
• 10. Selm VM, Jankowski NW. Conducting online surveys. Qual Quant 2006; 40: 435-56.
• 11. Gosling SD, Vazire S, Srivastava S, John OP. Should we trust web-based studies? A comparative analysis of six preconceptions about internet questionnaires. Am Psychol 2004; 59(2): 93-104.
• 12. Leece P, Bhandari M, Sprague S, Swiontkowski MF, Schemitsch EH, Tornetta P et al. Internet versus mailed questionnaires: A randomized comparison (2). J Med Internet Res 2004; 6(3): 30.
• 13. Eysenbach G, Wyatt J. Using the internet for surveys and health research. J Med Internet Res 2002; 4(2): 13.
• 14. Ritter P, Lorig K, Laurent D, Matthews K. Internet versus mailed questionnaires: a randomized comparison. J Med Internet Res 2004; 6(3): 29.
• 15. Schonlau M. Will web surveys ever become part of mainstream research? J Med Internet Res 2004; 6(3): 31.
• 16. Rankin KM, Rauscher GH, McCarthy B, Erdal S, Lada P, Il'yasova D, Davis F. Comparing the reliability of responses to telephone-administered web-based surveys in a case-control study of adult malignant brain cancer. Cancer Epidemiol Biomarkers Prev 2008; 17(10): 2639-46.
• 17. Kongsved SM, Basnov M, Holm-Christensen K, Hjollund NH. Response rate and completeness of questionnaires: a randomized study of internet versus paper-and-pencil versions. J Med Internet Res 2007; 9(3): 25.
• 18. Cook JV, Dickinson HO, Eccles MP. Response rates in postal surveys of healthcare professionals between 1996 and 2005: an observational study. BMC Health Services Research 2009; 9: 160.
• 19. Malin B, Barrowman N. Comparing administration of questionnaires via the Internet to pen-and-paper in patients with heart failure: randomized controlled trial. J Med Internet Res 2009; 11(1):e3.
• 20. Mazzon JA. O método de coleta de dados pelo correio: um estudo exploratório. In: Mazzon JA, Guagliardi JA, Fonseca JS. Marketing: Aplicações de métodos quantitativos São Paulo: Ed. Atlas, 1983.
• 21. Hipólito JAM. Como usar a Internet em pesquisa. In: I SEMEAD - Seminários e Administração. Programa de Pós-graduação em Administração, FEA-USP: São Paulo; 1996.
• 22. Silva SM, Santos C, Siqueira JO. O uso do questionário eletrônico na pesquisa acadêmica: Um caso de uso na Escola Politécnica da USP. II SEMEAD - Seminários e Administração. Programa de Pós-graduação em Administração, FEA-USP: São Paulo; 1997.
• 23. Vasconcelos L, Guedes LFA. E-surveys: Vantagens e Limitações dos Questionários Eletrônicos Via Internet no Contexto da Pesquisa Científica Apresentação realizada na X SEMEAD, FEA-USP, 2007. Disponível em: http://www.ead.fea.usp.br/Semead/10semead/sistema/resultado/an_resumo.asp?cod_trabalho=420 [Acessado em 26 de dezembro de 2010]
• 24. Noordman J, Van Dijk L, Friele R. Patient organizations and the reimbursement process for medicines: an exploratory study in eight European countries. BMC Health Serv Res 2010; 10(45).
• 25. Whitehead LC. Methodological and ethical issues in internet-mediated research in the field of health: An integrated review of the literature. Soc Sci Med 2007; 65(4): 782-91.

*

Sonia Mansoldo Dainesi was responsible for the study design, conduction of the survey on the internet, analysis and discussion of results, writing the manuscript; Moises Goldbaum, as the PhD project faculty supervisor was responsible for critical analysis of the project, including literature review, discussion of results and conclusions, and final review of the manuscript.

Publication Dates

• Publication in this collection
  20 Feb 2013
• Date of issue
  Dec 2012

History

• Received
  17 Aug 2011
• Accepted
  16 Mar 2012
• Reviewed
  17 Aug 2011