Traditional learning and problem-based learning: self-perception of preparedness for internship

Método tradicional e aprendizado baseado em problemas: autopercepção do preparo para o internato

ARTIGO ORIGINAL

Traditional learning and problem-based learning: self-perception of preparedness for internship

Método tradicional e aprendizado baseado em problemas: autopercepção do preparo para o internato

Laís Pereira Bueno Millan^I; Beatriz Semer^I; José Mauro da Silva Rodrigues^II; Reinaldo José Gianini^III

^IGraduate Students, School of Health and Medical Sciences, Pontifícia Universidade Católica de São Paulo (PUC-SP), São Paulo, SP, Brazil

^IIAssociated Professor, School of Health and Medical Sciences, PUC-SP, São Paulo, SP, Brazil

^IIIFull Professor, School of Health and Medical Sciences, PUC-SP; Researcher, LIM39 HC/FMUSP, São Paulo, SP, Brazil

^{Correspondence to}

SUMMARY

OBJECTIVE: This study aims to evaluate Pontificia Universidade Católica de São Paulo (PUC-SP) medical students' perception of their preparedness to attend the internship course by comparing students who entered the internship in 2009, who were taught according to the traditional learning method, and those who entered the internship in 2010, who were taught according to the new method, i.e. problem-based learning (PBL).

METHODS: 50 traditional learning method students answered a standard Lickert scale questionnaire upon entering internship in 2009. In 2010, the process was repeated with PBL students. The questionnaire was based upon the Preparation for Hospital Practice Questionnaire. This questionnaire was evaluated by professors from three medical schools in Brazil regarding its applicability. The original questions were classified according to the importance these professors attributed to them, and less important questions were removed. Scores obtained from the Student's t-test were considered significant with p < 0.05.

RESULTS: A significant statistical difference was observed in 16 questions, and the traditional learning method students reported higher average scores. When questions were divided into dimensions, a significant statistical difference appeared in the dimensions " social aspects of health", "medical skills", and "ethical concepts"; traditional learning method students again reported higher scores (p < 0.001 for all dimensions). Higher scores were also reported when the average of the answers to the whole questionnaire was calculated.

CONCLUSION: Traditional learning method students consider themselves to be better prepared for internship activities than PBL students, according to the following three comparative means: by analyzing the answers to each question, by grouping these answers into dimensions, and by calculating the means of answers to the whole questionnaire.

Keywords: Problem-based learning; curriculum; internship and medical residency; medical education; teaching.

RESUMO

OBJETIVO: Este estudo visa avaliar a percepção dos alunos de Medicina da Pontifícia Universidade Católica de São Paulo (PUC-SP) quanto ao seu preparo para cursar o internato, comparando os alunos que ingressaram no internato em 2009, no método tradicional, e aqueles que o iniciaram em 2010, no novo método, o aprendizado baseado em problemas (ABP).

MÉTODOS: Aplicou-se um questionário padronizado, respondido através de uma escala de Lickert, a cinquenta alunos do método tradicional, no início do internato, em 2009. Em 2010, o processo foi repetido com os alunos do novo método. O questionário foi baseado no Preparation for Hospital Practice Questionnaire. Esse questionário foi avaliado, no Brasil, quanto à sua aplicabilidade, por professores de três cursos de Medicina. As questões originais foram classificadas de acordo com a importância atribuída por eles e as consideradas menos relevantes foram retiradas. Compararam-se os resultados por meio do teste t de Student com p < 0,05.

RESULTADOS: Observa-se diferença estatística significante em dezesseis questões, sendo que os alunos do método tradicional apresentaram maiores médias. Quando se separam as questões em dimensões, nota-se diferença estatística significante em "aspectos sociais da saúde", "habilidades clínicas" e "conceitos éticos": os alunos do método tradicional demonstraram, novamente, maiores valores (p < 0,001 para todas as dimensões). Calculando-se a média de respostas do questionário inteiro, também observam-se maiores valores.

CONCLUSÃO: Conclui-se que os alunos do método tradicional se julgam melhor preparados para as atividades do internato do que os do ABP, nas três vias de comparação: analisando-se as respostas de cada questão, agrupando-as em dimensões e calculando-se a média de respostas do questionário inteiro.

Unitermos: Aprendizagem baseada em problemas; currículo; internato e residência; educação médica; ensino.

INTRODUCTION

Active teaching-learning methodologies, including problem- based learning (PBL), are a new education method employed in medical schools. Under this new method, students play a more active and fundamental role in their learning process. Educational tutoring is the most important activity: students are divided into small groups, consisting of approximately ten members, and monitored by an assisting medical tutor. In the beginning of the week, a certain problem is submitted to the students, and they subsequently discuss and raise questions concerning the case, and define the objectives for study. By the end of the week they meet again, and share their findings and observations. In addition to tutoring, PBL allows direct contact between students and patients from the beginning of their education process, through case studies at the hospital, under the preceptor's guidance.

McMaster University in Canada was a pioneer in the use of such methodology in the 1960's, which was subsequently adopted by Maastricht University, in the Netherlands^1,2. In 1985, the Harvard Medical School re-designed its curriculum based on PBL principles³. In Brazil, some medical schools, such as the Faculdade de Medicina de Marília (FAMEMA) and the Universidade Estadual de Londrina (UEL) have already adopted the PBL method.

Despite its broad introduction in several universities worldwide, this new method and its effectiveness are still open to discussion. The newly introduced PBL method is still controversial in the educational community, particularly in the medical field, and some articles have been published on this subject. One of these studies was conducted in Canada, in 2008, and consists of a meta-analysis in which several authors discuss the PBL method⁴.

In addition to the studies about PBL itself, other studies compare it to the traditional, expositive class-based, method. In 2006, in Australia, the opinions of interns from the University of Sydney were evaluated through the Preparation for Hospital Practice Questionnaire (PHPQ) regarding their weaknesses and strengths, and their impressions were compared to those of the traditional learning method students reported in a previous survey⁵. Another study with pediatrics residents made qualitative analyses regarding the time spent in self-directed studies by students undergoing PBL sessions, and compared them to the analyses of the time spent by students in lectures⁶.

The PBL method was implemented by the Pontifícia Universidade Católica de São Paulo (PUC-SP) six years ago; at the beginning of this study, the traditional method was only used in the fifth and sixth years of the medical course.

This study aimed to compare the perception of students who entered internship in 2009 (traditional learning method) to that of the students who entered internship in 2010 (PBL method) in order to ascertain their preparedness for internship.

METHODS

This study was approved by the Ethics Committee of the School of Health and Medical Sciences of PUC-SP. All participants signed an informed consent.

A questionnaire based on the PHPQ consisting of 40 questions was submitted to the students. The questions related to eight important issues that allow for the evaluation of the students' perception of their preparedness for internship activities⁵:

1. Interpersonal skills

2. Confidence and cooperation

3. Collaboration (bond amongst group members in order to care of patients)

4. Relationship with patients and practical skills

5. Scientific knowledge (such as knowledge of illness and of therapy)

6. Health prevention and promotion

7. Comprehensive concern (seeing the patient as a whole)

8. Self-directed study (evaluation of their own performance and identification of learning requirements).

First, the questionnaire was evaluated for its applicability, through a Lickert type scale with four scales, by professors of PUC-SP, FAMEMA and the Medical School of the Universidade de São Paulo (FMUSP). After the evaluation, nine questions were removed; those with more than 15 "moderately important" or "little important" answers. In addition to these questions, those with four or more "little important" or "not important" answers were also excluded.

Subsequently, the questionnaire was submitted to 50 fifth-year students admitted in 2009 (traditional learning method) and 50 fifth-year students admitted in 2010 (PBL method). The questionnaire consisted of 31 questions designed to classify, on a Lickert type scale with four scores (minimum score 1, maximum score 4), the students' perception of their ability to perform internship activities.

The Student's t-test was used for the statistical evaluation of results, with p < 0.05.

RESULTS

In order to facilitate the interpretation of data obtained, the answers were organized as follows: the average of answers given to each question by students from each curricular method was calculated as described in Table 1, and the most likely variation between results was presented (95% CI).

A significant statistical difference was observed in questions 1, 2, 6, 7, 8, 14, 15, 16, 18, 19, 23, 24, 25, 26, 27, and 31; traditional method students reported higher scores.

Additionally, the questions were also grouped in four dimensions:

1. Social aspects of health

2. Medical skills

3. Ethical concepts

4. Learning techniques

Subsequently, the average of the answers given by students in each dimension was calculated, as shown in Table 2.

A significant statistical difference was observed in the dimensions "social aspects of health", "medical skills", and "ethical concepts"; again, traditional learning method students reported higher scores.

DISCUSSION

Analyzing Table 1, it is clear that traditional learning method students demonstrated that they feel better prepared for internship activities when compared to PBL students in several questions, to wit: 1, 2, 6, 7, 8, 14, 15, 16, 18, 19, 23, 24, 25, 26, 27 and 31.

The same conclusion may be reached from the interpretation of data described in Table 2. When the questions are divided into dimensions, it can be concluded that students taught under the traditional learning method consider themselves to be better prepared in relation to "social aspects of health", "medical skills", and "ethical concepts"; no statistical difference could be ascertained in relation to "learning techniques" only. Furthermore, statistical significance could be ascertained when all answers were compared; traditional learning students considered themselves to be better prepared for the proposed activities.

The convenience sampling used in the study resulted in a statistical selection bias, which is explained by the difficulty to meet internship students, and thus questionnaires were applied in certain medical meetings attended by a great number of students. Therefore, it is possible that the sampling does not appropriately represent each group of students.

Table 3 shows a gender difference between both groups: in the traditional group, women's participation was higher, while in the PBL group the results were more homogeneous. This fact may have influenced research outcomes since women may mature faster than men. Regarding the age group of the students interviewed, the compared groups were similar: most were 23 or 24 years old, which slightly influenced the results obtained. It is worthwhile mentioning that none of the students had failed final exams.

Accordingly, several interpretations of the results can be made. Traditional learning students may have overestimated the values attributed to questions, since they were aware of the research purpose: a comparison between their learning method, which would be replaced, and a new, allegedly better, learning method.

On the other hand, questions about PBL's constitution failure may be raised. The strictness and the content of evaluations made on PBL method and its structural organization are also questionable. It is possible that PBL has not yet obtained the staff's effective adherence and participation.

It is also possible that students feel insecure towards the PBL method because it has been newly implemented. The fifth-year students admitted in 2010 were the first to be taught under the new method, which is still subject to changes for improvement purposes.

According to Koh, PBL undergraduate students are used to working in a poorly structured education environment and, therefore, are encouraged to work independently. Furthermore, they often work in small groups, which encourages them to develop good communication among themselves. These capabilities may produce positive results while they perform internship activities⁷. Nevertheless, in the "medical skills" dimension of this survey, it must be observed that PBL students feel less confident when compared to traditional learning method students. PBL students may feel insecure exactly because they are required to work on an independent basis instead of playing a secondary role in the learning process, since under the traditional method most of the curricular content is taught through expositive classes.

Moreover, concepts (satisfactory and unsatisfactory) are attributed to (a few) formal evaluations instead of numerical grades, under the new method. Thus, students may have greater difficulties in evaluating whether the skills they have acquired are sufficient to practice medicine, which may contribute to their lack of confidence. The self evaluation is an often repeated situation since in all educational tutoring meetings students critique their own performance and that of the group, aiming to improve study dynamics. Because they are constantly evaluating themselves, which does not occur in the traditional method, PBL students might view their performance in a different manner.

Students may also develop a feeling that the traditional method provides better education when compared to PBL method. Their confidence would be provided by teachers, who represent the authority within the academic environment by providing guidance, delimiting and securing the learning process.

On the other hand, several studies strongly suggest that PBL students are better prepared for medical practice, according to their supervisors' evaluation, as explained in Neville's meta-analysis⁴. However, PBL students' uncertainties apprehended from the current study are translated into a paradox, which may be understood through the idea suggested by Norman. Undergraduate PBL students are subject to undergoing uncertain situations and on a long-term basis they may develop the ability to deal with such difficulties, which is a very important tool for the medical practice⁸. Additionally, through the experience they get from the medical practice, these students develop a critical profile, which may affect their self-evaluation. Therefore, an intern that feels insecure in the face of challenges is given the opportunity to mature within a learning environment before becoming a doctor.

The literature lacks information that would allow for a proper comparison between both learning methods in relation to students' admission to internship. This may be explained by the fact that there is only one ideal occasion for comparing both groups: it is when the last traditional course group and the first PBL course group enter internship. After this occasion, the comparison between both groups is less reliable because of the temporal distance from the emotional experience the students had. However, notwithstanding this difficulty, further studies about the issue are needed. It would also be appropriate to correlate the results obtained in this study with formal evaluations conducted by preceptors, since the limitation here was the ascertainment of the students' preparation for internship solely from the students' point of view.

It is important to point out that the subjective experiences of each individual strongly influence their perception of their own performance. Therefore, the complex self-criticism process translates into a limitation to the interpretation of the results obtained.

It is important to outline that although feeling confident, students may not be actually able to put into practice all things they intend to; confidence and knowledge are not synonymous.

CONCLUSION

This study has demonstrated that traditional learning method students consider themselves to be better prepared for internship activities than PBL method students. This is evident in three degrees of comparison: by analyzing the answers to each question, by grouping these answers into dimensions, and by calculating the average of answers of the whole questionnaire.

Submitted on: 02/07/2012

Approved on: 04/08/2012

Conflict of interest: None.

Study conducted at Pontifícia Universidade Católica de São Paulo (PUC-SP) São Paulo, SP, Brazil

References

• 1. McMaster University. Canada: About McMaster. [cited 2011 jun 15]. Available from: http://www.mcmaster.ca/welcome/aboutmac.cfm
• 2. Maastrich University. Netherlands: history - Maastrich University; 2010. [cited 2011 jun 5]. Available from: http://www.maastrichtuniversity.nl/web/Main/AboutUM/History.htm
• 3. Lowry S. Making change happen. BMJ. 1993;306:320-22.
• 4. Neville AJ. Problem-based learning and medical education forty years on. Med Princ Pract. 2009;18:1-9.
• 5. Dean SJ, Barrat AL, Hendry GD, Lyon PMA. Preparedness for hospital practice among graduates of a problem-based, graduate-entry medical program. Med J Aust. 2003;178:163-6.
• 6. Ozuah PO, Curtis, J, Stein REK. Impact of problem-based learning on resident's self-directed learning. Arch Pediatr Adolesc Med. 2001;155:669-72.
• 7. Koh GC, Khoo HE, Wong ML, Koh D. The effect of problem-based learning during medical school on physician competency: a systematic review. Can Med Assoc J. 2008;178:34-41.
• 8. Norman G. Problem-based learning makes a difference. But why? Can Med Assoc J. 2008;178:61-62.

Publication Dates

• Publication in this collection
  17 Oct 2012
• Date of issue
  Oct 2012

History

• Received
  07 Feb 2012
• Accepted
  08 Apr 2012