Development and assessment of a multimedia computer program to teach pleural drainage techniques

Abstracts

OBJETIVO: Desenvolver um programa educacional de computador sobre drenagem pleural voltado a profissionais de saúde, com recursos de multimídia, e avaliar sua eficácia com alunos. MÉTODOS: Foi planejado o desenvolvimento do programa e a avaliação foi realizada com 35 alunos do curso de medicina divididos aleatoriamente em dois grupos. O grupo 1, composto por 18 alunos, estudou com o programa e o grupo 2, com 17 alunos, recebeu uma aula teórica tradicional, com professor experiente. Os alunos do grupo 1 foram submetidos a duas avaliações subjetivas por questionários, e os alunos de ambos os grupos foram submetidos a uma prova teórica objetiva com testes de múltipla escolha e questões descritivas. Os resultados da prova teórica foram comparados por meio do teste de Mann-Whitney. RESULTADOS: A avaliação subjetiva quanto aos aspectos de informática e conteúdo mostrou resultados entre ótimo, muito bom e bom. O programa foi considerado totalmente didático por 16 alunos (88,9%) e 17 alunos (94,4%) responderam que pode vir a substituir parcialmente as aulas tradicionais. Não houve diferença significante entre os dois grupos nos testes de múltipla escolha, mas houve diferença significante nas questões descritivas (p < 0,001). O grupo 1 obteve notas maiores que as do grupo 2. CONCLUSÕES: O desenvolvimento do programa de computador para ensino de drenagem pleural na Universidade Federal de São Paulo/Escola Paulista de Medicina mostrou-se factível. A avaliação subjetiva deste novo método de ensino mostrou-se altamente satisfatória e a avaliação objetiva mostrou que o programa foi tão eficaz quanto o ensino tradicional.

Ensino; Cirurgia torácica; Multimídia; Drenagem; Doenças pleurais

ORIGINAL ARTICLE

Development and assessment of a multimedia computer program to teach pleural drainage techniques^*

João Aléssio Juliano Perfeito^I; Vicente Forte^II; Roseli Giudici^III; José Ernesto Succi^IV; Jae Min Lee^V; Daniel Sigulem^VI

^IAdjunct Professor in the Department of Thoracic Surgery. Escola Paulista de Medicina da Universidade Federal de São Paulo - UNIFESP-EPM, Federal University of São Paulo Paulista School of Medicine - São Paulo, Brazil

^IIAssociate Professor in the Department of Thoracic Surgery. Escola Paulista de Medicina da Universidade Federal de São Paulo - UNIFESP-EPM, Federal University of São Paulo Paulista School of Medicine - São Paulo, Brazil

^IIIAdjunct Professor in the Department of Thoracic Surgery. Escola Paulista de Medicina da Universidade Federal de São Paulo - UNIFESP-EPM, Federal University of São Paulo Paulista School of Medicine - São Paulo, Brazil

^IVAssistant Professor in the Department of Thoracic Surgery. Escola Paulista de Medicina da Universidade Federal de São Paulo - UNIFESP-EPM, Federal University of São Paulo Paulista School of Medicine - São Paulo, Brazil

^VPhysician in the Department of Health Information. Escola Paulista de Medicina da Universidade Federal de São Paulo - UNIFESP-EPM, Federal University of São Paulo Paulista School of Medicine - São Paulo, Brazil

^VIFull Professor in the Department of Health Information. Escola Paulista de Medicina da Universidade Federal de São Paulo - UNIFESP-EPM, Federal University of São Paulo Paulista School of Medicine - São Paulo, Brazil

Correspondence to

ABSTRACT

OBJECTIVE: To develop a multimedia educational computer program designed to teach pleural drainage techniques to health professionals, as well as to evaluate its efficacy.

METHODS: We planned and developed a program, which was evaluated by 35 medical students, randomized into two groups. Group 1 comprised 18 students who studied using the program, and group 2 comprised 17 students who attended a traditional theoretical class given by an experienced teacher. Group 1 students were submitted to two subjective evaluations using questionnaires, and both groups took an objective theoretical test with multiple-choice questions and descriptive questions. The results of the theoretical test were compared using the Mann-Whitney test.

RESULTS: The subjective evaluation of the technological aspects and content of the program ranged from excellent to very good and good. The software was considered highly instructive by 16 students (88.9%), and 17 students (94.4%) thought it might partially substitute for traditional classes. Between the two groups, there was no significant difference in the multiple-choice test results, although there was such a difference in the descriptive question results (p < 0.001), group 1 students scoring higher than did those in group 2.

CONCLUSIONS: The computer program developed at the Federal University of São Paulo Paulista School of Medicine proved to be a feasible means of teaching pleural drainage techniques. The subjective evaluation of this new teaching method revealed a high level of student satisfaction, and the objective evaluation showed that the program was as efficacious as is traditional instruction.

Keywords: Teaching; Thoracic surgery; Multimedia; Drainage; Pleural diseases.

Introduction

The procedure of placing a tube in the pleural cavity has been described since the time of Hippocrates and has been improved upon over time. In this process, the development of the water-seal system was a great advance. However, specific knowledge is required in order to perform the procedure, as well as to care for and monitor patients submitted to it.

Computer science had its beginning in 1946. In 1959, the first systems for medical decision-making were developed.⁽¹⁾ The first hospital-based health care systems began to appear in 1962. ⁽²⁾ In the 1970s, with the advent of the microcomputer, information technology began to be used in medicine, in administrative and hospital systems as well as in information management,⁽³⁾ systems that later evolved to include the physician-patient relationship and teaching. Simultaneously, it became evident that the application of the new technology had advantages for a variety of sectors, with legible clinical charts and computer-based programs to aid clinical decision-making.^(4,5)

The potential use of information technology in teaching began to be questioned.^(6,7) Comparisons were made between traditional teaching and computer-based teaching.^(8-13) The cost of this new way of teaching was investigated, and the computer came to seem more and more advantageous over time.⁽¹⁴⁾

Computerized simulations came to be used in medical education, and the results were satisfactory.^(15-17) The changes and advantages that electronics would bring to education were discussed.⁽¹⁸⁾

At the Escola Paulista de Medicina (EPM, Paulista School of Medicine), educational programs were in development as early as 1990. ^(19,20) The incorporation of sounds and the storage of greater amounts of information in formats such as the compact disc read-only memory (CD-ROM) constituted a great advance.^(21,22) A methodology for the development of multimedia projects in the area of medicine was delineated, and the results were satisfactory.⁽²³⁾ The concept of distance learning emerged and began to grow at the Universidade Federal de São Paulo (UNIFESP, Federal University of São Paulo), leading to the creation of the Virtual UNIFESP.

The systems, materials and precautions related to pleural drainage should be known by all health professionals.^(24-27) Taking into consideration how easily new generations learn using information technology, we had the idea to use this new tool in order to disseminate our experience with pleural drainage.

The objective of this study was to develop a multimedia educational computer program designed to teach pleural drainage techniques to health professionals, as well as to evaluate its efficacy in teaching undergraduate medical students.

Methods

Development

We planned and developed an educational program designed to teach pleural drainage techniques. Acquisition of knowledge was defined based on articles published in journals and books, as well as on our teaching experience.^(24-26)

The program uses multimedia resources: original drawings; photographs; animation; videos; texts; and narratives. A microcomputer was used in the data processing. Macromedia Director 6.5 was the authorship program used, and video images were edited using Quicktime 3.0 program for Windows. The program was finalized and recorded onto a CD-ROM.

Assessment

After the study protocol had been approved by the ethics committee, the program was tested with fourth-year undergraduate UNIFESP-EPM medical students who were interns in thoracic surgery. The target public for the test was the third class in the curricular training program. The class comprised 36 students. Without any prior warning, the class was divided into two similar groups by random drawing. The students in group 1 studied independently using the program, whereas those in group 2 attended a traditional theoretical class.

The theoretical class on pleural drainage was given by a professor in the thoracic surgery department with considerable experience in the subject and whose teaching skills are recognized throughout the university. This professor had no knowledge of the program.

The assessment was divided into two parts: the subjective assessment, in which two questionnaires were applied to group 1 students; and the objective assessment, which was applied to both groups.

The objective assessment consisted of a theoretical test prepared by a board of three physicians and professors in the Department of Thoracic Surgery, none of whom were familiar with the program. The board was informed that the objective was not to grade the students but rather to compare the two methods. The objective theoretical test comprised 36 multiple-choice questions and 7 descriptive questions.

On the date scheduled for the class, the experiment was conducted. The group randomly selected for the theoretical class remained in the classroom, and the computer study group was sent to the computer room. One student in group 2 (traditional class) was late and was thereby excluded from the experiment. Therefore, there were 18 students in group 1 and 17 students in group 2, totaling 35 students.

The preestablished time for the theoretical class was one hour and thirty minutes (the usual duration of an undergraduate class), and group 1 students were allowed to study independently using the program for the same length of time. Immediately after the end of the computer study, group 1 students filled out the first assessment, designated short subjective assessment, with questions aimed at detecting their first impression. Subsequently, an objective theoretical test was administered to both groups, with an unlimited time period in which to answer. Finally, a detailed subjective assessment was performed by group 1 students by filling out a second, more detailed questionnaire.

The objective theoretical test was corrected by the board that prepared it. The board assigned a score of 0.2 to each correct multiple-choice question and a maximum score of 0.4 to each descriptive question. The maximum possible grade would be 7.2 on the multiple-choice section and 2.8 on the descriptive section, totaling 10.

The results of the objective theoretical test-scores on the multiple-choice and descriptive sections, as well as overall grade-were tabulated for the two groups. Results were analyzed using the Mann-Whitney test⁽²⁸⁾ in order to compare groups 1 and 2. The level of statistical significance required to reject the null hypothesis was set at 0.05 or 5%.

Results

Development

Developing an educational computer program designed to teach pleural drainage techniques proved feasible. The program was created and burned onto a CD-ROM, occupying 214 megabytes of storage space. The theoretical content was divided into five sections: Introduction, Fundamentals, Materials, Technique and Evolution. At the bottom of the main screen, in all parts of the program, the user sees navigation buttons, with the option of reading the instructions and returning to the initial screen. In addition, the content of each section is narrated.

Drawings, photographs and animation are used in the Introduction, which is subdivided into the following parts: Anatomy (Figure 1); Pleural puncture; and Pleural drainage.

The Fundamentals section uses animated drawings and is subdivided into Open pneumothorax, Basic system for closed pleural drainage, Oscillation of the fluid level, Fluid drainage, Air drainage, Diagnosis of fistula and Open pleural drainage.

The Materials section uses photographs and drawings. Small photographs of the items used in pleural drainage are displayed; the user can select each of the 18 items using the mouse (Figure 2).

The Technique section comprises narrated videos and is subdivided into the topics related to pleural drainage performed with chest tube.

The Evolution section uses photographs, drawings and animation to describe the proper care of the drainage system and of the patients in whom it is employed. It is subdivided into Care, Continuous aspiration and Tube removal.

Assessment

Short subjective assessment

The short subjective assessment comprised six statements, and students chose one of the following options: strongly disagree; disagree; neither agree nor disagree; agree; or strongly agree. The statements were as follows:

• The program is very user-friendly.

• The program is highly enjoyable.

• The program offers immediate feedback.

• The program is highly didactic.

• I would like to have access to similar programs on other topics.

• Programs like this could effectively replace teachers.

The answers given by the 18 students are shown, statement by statement, in Figure 3.

Detailed subjective assessment

The detailed subjective assessment comprised a questionnaire on three topics: Information technology aspects; Theoretical content; and Overall assessment of the program.

Information technology aspects

The wording of and the respective answers to the questions in this section were as follows:

• "How would you evaluate the presentation of the program (the visual aspect of the screens, images and animation)?"

Answers: excellent, 8 students (44.4%); very good, 8 students (44.4%); good, 2 students (11.1%); fair, 0 students (0%); poor, 0 students (0%).

• "How easy-to-navigate is the program?"

Answers: excellent, 9 students (50%); very good, 8 students (44.4%); good, 0 students (0%); fair, 1 student (5.6%); poor, 0 students (0%).

• "Was it easy to find the information you were seeking?"

Answers: excellent, 3 students (16.7%); very good, 10 students (55.6%); good, 4 students (22.2%); fair, 1 student (5.6%); poor, 0 students (0%).

• "How would you rate the navigation buttons of the program?"

Answers: excellent, 5 students (27.8%); very good, 12 students (66.7%); good, 0 students (0%); fair, 1 student (5.6%); poor, 0 students (0%).

• "Did you have any problems using the program?"

Answers: no, 15 students (83.3%); yes, 3 students (16.7%).

(It should be clarified that all of the problems reported were related to the computer rather than the program.)

• "How would you classify your level of knowledge in information technology?"

Answers: excellent, 0 students (0%); very good, 3 students (16.7%); good, 10 students (55.6%); fair, 5 students (27.8%); poor, 0 students (0%).

Theoretical content

The wording of and the respective answers to the questions in this section were as follows:

• "Did the theoretical content meet your expectations?"

Answers: excellent, 2 students (11.1%); very good, 8 students (44.4%); good, 6 students (33.3%); fair, 2 students (11.1%); poor, 0 students (0%).

• "How would you evaluate the way the content was divided and distributed in the program?"

Answers: excellent, 7 students (38.9%); very good, 6 students (33.3%); good,5 students (27.8%); fair, 0 students (0%); poor, 0 students (0%).

• "How would you evaluate the content of the Introduction?"

Answers: excellent, 5 students (27.8%); very good, 7 students (38.9%); good, 6 students (33.3%); fair, 0 students (0%); poor, 0 students (0%).

• "How would you evaluate the content of the Fundamentals section?"

Answers: excellent, 7 students (38.9%); very good, 4 students (22.2%); good, 7 students (38.9%); fair, 0 students (0%); poor, 0 students (0%).

• "How would you evaluate the content of the Materials section?".

Answers: excellent, 6 students (33.3%); very good, 8 students (44.4%); good, 3 students (16.7%); fair, 1 student (5.6%); poor, 0 students (0%).

• "How would you evaluate the content of the Technique section?"

Answers: excellent, 9 students (50%); very good, 6 students (33.3%); good, 3 students (16.7%); fair, 0 students (0%); poor, 0 students (0%).

• "How would you evaluate the content of the Evolution section?"

Answers: excellent, 6 students (33.3%); very good, 5 students (27.8%); good, 6 students (33.3%); fair, 1 student (5.6%); poor, 0 students (0%).

• "How would you evaluate the theoretical aspect of the program?"

Answers: excellent, 1 student (5.6%); very good, 11 students (61.1%); good,4 students (22.2%); fair,2 students (11.1%); poor, 0 students (0%).

Overall assessment of the program

The wording of and the respective answers to the questions in this section were as follows:

• "Do you think the CD-ROM was didactic?"

Answers: Completely, 16 students (88.9%); partially, 2 students (11.1%); hardly at all, 0 students (0%).

The frequencies of the answers to this question are presented in Figure 4.

• "How satisfied were you with the CD-ROM?"

Answers: Completely, 12 students (66.7%); partially, 6 students (33.3%); hardly at all, 0 students (0%).

• "Do you believe that CD-ROM programs can replace traditional theoretical classes?"

Answers: Completely, 1 student (5.6%); partially, 17 students (94.4%); not really, 0 students (0%).

• "Had you previously used a CD-ROM to learn about any medical subject?"

Answers: yes, 11 students (61.1%); no, 7 students (38.9%).

• "After using the CD-ROM on pleural drainage, how has your opinion changed, if at all, regarding the use of the CD-ROMs in medical education?"

Answers: More favorable, 14 students (77.8%); no change, 4 students (22.2%); less favorable, 0 students (0%).

Objective theoretical test

Table 1 shows the grades given to group 1 students and group 2 students on the descriptive section of the test, with respective means, as well as the overall grades.

When the group 1 grades on the multiple-choice section were compared to those of group 2, the calculated Z score was found to be 0.97 for a critical Z score of 1.96. Therefore, there was no significant difference between the two groups.

When the two groups were compared in terms of their grades on the descriptive section of the test, the calculated Z score was found to be 3.33 for a critical Z score of 1.96. Therefore, there was a significant difference (p < 0.001), the group 1 grades being significantly higher than those of group 2.

When the overall grades for group 1 were compared with those for group 2 using the Mann-Whitney test, the calculated Z score was found to be 1.9 for a critical Z score of 1.96. Therefore, there was no significant difference between the two groups.

Discussion

Finding the best method to motivate the student and impart knowledge has been moving the world for centuries. In the field of surgery, the physician is confronted with the need to approximate the binomial of surgery as science and surgery as art. We cultivate the scientific mentality of modern times. However, we are also the heirs of lay practitioners and barber-surgeons of the middle ages. Surgeons will only be complete when they learn to be humanists, scientists and artists. Becoming a true surgeon means learning from books, theoretical classes, discussion groups, as well as operating room observations and problem-based instruction.^(6,29)

The role of the teacher is also being revised. Finding the right ratio between teaching and knowledge acquisition is not an easy task, and this is where new resources for the transmission of information should step in.^(7,12,15,30)

The major objective of our study was to develop an educational program to teach pleural drainage techniques to surgeons. We tried to use a number of different resources so that the final product would not become monotonous, thus turning the student from a passive recipient of information into an active seeker of knowledge.

In our study, we divided the test into subjective and objective assessment according to the validation trend of most educational programs.^{(12-14,16,17)}

Regarding the sample, we were aware that the credibility of the test would depend on the students not previously knowing that they were going to be tested, especially because the subject was related to technological innovation, which could attract sympathetic volunteers involved in new technologies. Although this is a new paradigm, we have to compare it to the traditional model. However, we realize that changes need much more acceptance from society than simple proof that what is new is better.

The short subjective assessment format carried out immediately after the study by computer revealed a great majority of satisfactory answers to the questions on whether the program is user-friendly, pleasant, whether it offered immediate feedback, whether it is didactic, and whether the students would like to have programs on other topics. To these, we added a thought-provoking question. We wondered what this generation of students thought about the feasibility of replacing teachers with computer programs, although that is certainly not the principal argument of this study. We think that the role of the teacher is not to merely impart information. In our opinion, imparting experience and setting an example are functions that cannot be replaced. From the tests results, it seems that this is also the point of view of the students. A similar response was found by another author.⁽²²⁾

The detailed subjective assessment applied immediately after the objective theoretical test aimed at allowing the student to more calmly evaluate and help to develop the future versions of the program, without the stress that precedes tests. We did not have a clear idea of the degree to which the students were knowledgeable regarding information technology. Ten students (55.6%) classified their knowledge of the subject as good, and this is the target public that the teacher of the future should be prepared for.

The assessment of the theoretical content was also gratifying. The vast majority of the students answered that both format and content were excellent, very good or good. The third part of the detailed subjective assessment concerned the overall assessment of the program. The first question was about the teaching capacity of the program, and the assessment was very satisfactory: 16 students (88.9%) answered that it was completely didactic. Here, once again, the student could calmly answer whether the program could ever replace traditional theoretical classes, and the great majority, 17 students (94.4%), answered that it could only partially replace such classes. This seemed to us as an argument to be added to our statement: the teacher is irreplaceable in imparting experience, but can and should be aided by other methods of imparting knowledge. When asked about whether they had become more in favor of this method in medical education after using the program, 14 students (77.8%) stated that they had, and none of the students stated that they had become less in favor, which encourages us to continue working in this area.

An acceptable test of the program depended on the test being blind. Therefore, neither the group of three professors who prepared the objective test nor the professor who applied it was familiar with the program. In terms of the results on the multiple-choice section of the test, there was no significant difference between the grades of students attending the class given by the professor and those of students studying with the computer program. However, when we analyzed the descriptive section of the test, we found that the students studying with the program achieved significantly better results than did the students attending the theoretical class. Therefore, computer-based study might lead to better assimilation of knowledge, due to the nature of multimedia resources and to the possibility of immediately and individually going back a step when necessary. This improved assimilation of knowledge might be more advantageous when writing out an answer rather than when simply choosing the best alternative. The difference between the groups regarding the grades on the descriptive section was not significant and did not influence the overall grades on the test, which were comparable between the two groups.

We believe that the role of the teacher of the future will be to educate rather than simply to inform, and that the teacher will be fundamental in the practical formation of the student, which means not merely teaching how to handle instruments but rather to reflect on the values that characterize real physicians and on the posture taken in relating to patients in distress. It will be up to the teacher to teach, which is defined as perfecting the acquired knowledge, developing new methods and setting a good example. The teacher will be a tutor of their students, an extremely noble function, much nobler than that of simply repeating acquired information, since this function can be improved upon by new learning methods like the one developed and tested here.

In view of these results, we can conclude that teaching pleural drainage via a computer program in feasible. In addition, the subjective assessment of this new teaching method showed that the degree of satisfaction with the program was high among a sample of fourth-year medical students. Furthermore, the objective assessment demonstrated that the computer program was as efficacious as is traditional teaching. Moreover, students using the computer program produced significantly higher scores on the descriptive questions than did those taught using traditional methods.

References

• 1. Ledley RS, Lusted LB. Reasoning Foundations of medical diagnosis. In: Reggia JA, Tuhrim S, editors. Computer-assisted medical decision making. New York: Springer-Verlag; 1985. p. 46-79.
• 2. Blois MS, Shortliffe EH. The computer meets medicine: emergence of a discipline. In: Shortliffe EH, Perreault LE, Wiederhold G, Fagan LM, editors. Medical informatics: computer applications in health care. Reading, Mass: Addison-Wesley Pub; 1990. p. 1-36.
• 3. Degoulet P, Fieschi M. Hospital information systems. In: Degoulet P, Fieschi M, editors. Introduction to clinical informatics. New York: Springer; 1997. p. 91-104.
• 4. Johnston ME, Langton KB, Haynes RB, Mathieu A. Effects of computer-based clinical decision support systems on clinician performance and patient outcome. A critical appraisal of research. Ann Intern Med. 1994;120(2):135-42.
• 5. Drazen EL, Metzger JB, Ritter JL, Schneider MK, editors. Patient care information systems: successful design and implementation. New York: Springer; 1995. 218p.
• 6. Rosato FE. Self-assessment and continuing medical education. Surgery. 1972;71(4):642-4.
• 7. Fonkalsrud EW. Motivation of university surgeons to achieve excellence in teaching. Surgery. 1977;81(6):613-8.
• 8. Clayden GS, Wilson B. Computer-assisted learning in medical education. Med Educ. 1988;22(5):456-67.
• 9. Henry JB. Computers in medical education: information and knowledge management, understanding and learning. Hum Pathol. 1990;21(10):998-1002.
• 10. Andrews PV, Schwarz J, Helme RD. Students can learn medicine with computers. Evaluation of an interactive computer learning package in geriatric medicine. Med J Aust. 1992;157(10):693-5.
• 11. Webber WB, Summers AN, Rinehart GC. Computer-based multimedia in plastic surgery education. Plast Reconstr Surg. 1994;93(6):1290-300.
• 12. Rogers DA, Regehr G, Yeh KA, Howdieshell TR. Computer-assisted learning versus a lecture and feedback seminar for teaching a basic surgical technical skill. Am J Surg. 1998;175(6):508-10.
• 13. Teichman JM, Richards J. Multimedia to teach urology to medical students. Urology. 1999;53(2):267-70.
• 14. Dewhurst DG, Hardcastle J, Hardcastle PT, Stuart E. Comparison of a computer simulation program and a traditional laboratory practical class for teaching the principles of intestinal absorption. Am J Physiol. 1994;267(6 Pt 3):S95-104.
• 15. Sinclair MJ, Peifer JW, Haleblian R, Luxenberg MN, Green K, Hull DS. Computer-simulated eye surgery. A novel teaching method for residents and practitioners. Ophthalmology. 1995;102(3):517-21.
• 16. Waugh RA, Mayer JW, Ewy GA, Felner JM, Issenberg BS, Gessner IH, et al. Multimedia computer-assisted instruction in cardiology. Arch Intern Med. 1995;155(2):197-203.
• 17. Elves AW, Ahmed M, Abrams P. Computer-assisted learning; experience at the Bristol Urological Institute in the teaching of urology. Br J Urol. 1997;80(Suppl 3):S59-S62.
• 18. Marescaux J, Mutter D, Soler L, Vix M, Leroy J. [The virtual university applied to telesurgery: from tele-education to telemanipulation] [Article in French]. Chirurgie. 1999;124(3):232-9.
• 19. Sigulem D. Um novo paradigma de aprendizado na prática médica da UNIFESP/EPM [Thesis - Tenure]. São Paulo: Universidade Federal de São Paulo, 1997.
• 20. Cal RG, Moura Jr. JM, Danna Neto L, Carvalho AC, Rocha AL, Schor N, et al. Renal physiology educational program. Library of computer-assisted medical educational programs. In: Van Bemmel J, Zvarova, editors. Knowledge, information and medical education. 1 ed. North Holland: Elsevier Science; 1991. p. 198-202.
• 21. Lee JM, Koide MM, Salvador EM, Moura J, Ramos MP, Uras R, et al. Educational program on ophthalmology. Medinfo. 1995;8( Pt 2):1241-2.
• 22. Seabra D, Srougi M, Baptista R, Nesrallah LJ, Ortiz V, Sigulem D. Computer aided learning versus standard lecture for undergraduate education in urology. J Urol. 2004;171(3):1220-2.
• 23. Bernardo V. Metodologia para desenvolvimento de projeto multimídia aplicado ao ensino da medicina [dissertation]. São Paulo: Universidade Federal de São Paulo; 1996.
• 24. Perfeito JA, Crotti PL, Succi JE, Leao LE. Procedimentos de urgência em cirurgia torácica. In: Burihan E, editor. Emergências em cirurgia. São Paulo: Sarvier; 1995. p. 125-36.
• 25. Perfeito JA. Drenagem Pleural. In: Frisoli Jr A, Lopes CA, Amaral JL, Ferraro JR, Blum VF, editors. Emergencias: manual de diagnostico e tratamento. Sao Paulo: Sarvier; 1995. p. 583-7.
• 26. Perfeito JA, Schettini ST. Punção e drenagem pleural e pericárdica. In: Matsumoto T, editors. Terapia intensiva pediatrica. 2^nd ed. São Paulo: Atheneu; 1997. p. 1097-104.
• 27. Perfeito JA. Punção e drenagem pleural. Diag Trat. 1998;3(3):45-52.
• 28. Siegel S, Castellan NJ, editors. Nonparametric statistics for the behavioral sciences. New York: McGraw-Hill; 1988.
• 29. Schwartz RW, Donnelly MB, Young B, Nash PP, Witte FM, Griffen WO Jr. Undergraduate surgical education for the twenty-first century. Ann Surg. 1992;216(6):639-47.
• 30. Andrade JC. Nova metodologia para ensino e ensaio de técnicas operatórias em cirurgia cardíaca. [thesis]. São Paulo: Universidade Federal de São Paulo; 1994.

*

Trabalho realizado na Escola Paulista de Medicina da Universidade Federal de São Paulo - UNIFESP-EPM - São Paulo (SP) Brasil.

Publication Dates

• Publication in this collection
  01 Aug 2008
• Date of issue
  July 2008

History

• Accepted
  26 Sept 2007
• Received
  24 May 2007