Assessment of clinical skills in cardiovascular physiotherapy: Objective Structured Clinical Examination (OSCE)

Abstract

Introduction:

It is crucial to integrate realistic assessment approaches for cardiovascular physiotherapeutic skills in the professional context. The Objective Structured Clinical Examination (OSCE) is notably reliable and valid. However, understanding the student's perception after this experience is important for readjusting the assessment.

Objective:

To evaluate the strengths and limitations of applying the OSCE to undergraduate students during the Supervised Cardiovascular Physiotherapy Internship.

Methods:

Descriptive, qualitative, observational, and cross-sectional study. The students were submitted to the OSCE. After completion, a questionnaire was administered to assess their perception, containing seven items: A) General exam organization; B) Clinical vignettes and instructions; C) Adequacy of the tasks required; D) Quality of post-examination explanations; E) Complexity of the content involved, in which the student could assign a grade from 1 to 5 (1 = insufficient; 2 = fair; 3 = good; 4 = great; 5 = excellent); F) Difficulty with time management; and G) Emotional stress. Items F and G were rated: 1 = very low; 2 = low; 3 = moderate; 4 = high; and 5 = very high.

Results:

Fifty-one students responded to the questionnaire. For items A, B, C, D, and E, the most prevalent response was excellent (72.5%, 43.1%, 52.9%, 88.2%, and 54.9% respectively). Regarding item F, 54.9% of students reported moderate difficulty managing their time. Regarding item G, 94.2% reported a very high level of emotional stress.

Conclusion:

Application of the OSCE provided experiences of clinical cases with adequate complexity, in an organized manner and with an enriching final evaluation.

Keywords:
Checklist; Clinical skills; Objective Structured Clinical Examination; OSCE

Resumo

Introdução:

É crucial integrar abordagens avaliativas realistas para habilidades fisioterapêuticas cardiovasculares no âmbito profissional. O Objective Structured Clinical Examination (OSCE) é notoriamente confiável e válido, entretanto, compreender a percepção do aluno após essa experiência é importante para a readequação dessa avaliação.

Objetivo:

Avaliar os pontos fortes e as limitações da aplicação do OSCE em discentes do curso de graduação em fisioterapia durante estágio supervisionado de fisioterapia cardiovascular.

Métodos:

Trata-se de um estudo descritivo, qualitativo, observacional e transversal. Os discentes foram submetidos ao OSCE e, após a finalização, aplicou-se um questionário para avaliação da percepção contendo sete itens: A) organização geral do exame; B) casos clínicos e instruções para o exame; C) adequação das tarefas exigidas; D) qualidade das explicações pós-exame; E) complexidade do conteúdo envolvido, em que o discente poderia atribuir uma nota de 1 a 5 (1 = insuficiente; 2 = regular; 3 = bom; 4 = ótimo; 5 = excelente); F) dificuldade com o gerenciamento do tempo; e G) estresse emocional. Os itens F e G tinham as notas: 1 = muito baixo; 2 = baixo; 3 = médio; 4 = alto; e 5 = muito alto.

Resultados:

Cinquenta e um alunos responderam ao questionário. Para o itens A, B, C, D e E, a resposta mais prevalente foi excelente (72,5%, 43,1%, 52,9%, 88,2% e 54,9% respectivamente). Em relação ao item F, 54,9% dos alunos relataram dificuldade média a muito alta em gerenciar o tempo. Em relação ao item G, 94,2% relataram nível muito alto de estresse emocional.

Conclusão:

A aplicação do OSCE propiciou vivências de casos clínicos com complexidade adequada de forma organizada e com avaliação final enriquecedora.

Palavras-chaves:
Checklist; Habilidades clínicas; Exame Clínico Objetivo Estruturado; OSCE

Introduction

The Objective Structured Clinical Examination (OSCE) is one of the most valid, rational, effective, and reliable tests for evaluating clinical skills,¹ and it has been used to assess competencies in health-related education professions worldwide.²

In a typical OSCE, students engage in scenarios with actors trained by the responsible professor in each area, where they perform various clinical tasks in situations that simulate professional daily life. The responsible professor evaluates each student's reasoning and patient interaction. This assessment allows for the verification of a variety of skills in problem-solving, decision-making, and application of therapeutic techniques.³ Additionally, by incorporating diverse scenarios, students gain multiple skills and can encounter realistic situations that they will face when starting their professional practice, as in any educational experience within a supervised internship.⁴

The OSCE is structured by the faculty, aiming to diversify evaluative methods and its applicability has been evaluated in various healthcare settings; however, understanding the student's perspective in this context is important to guide educators and enhance this method in the field of physiotherapy. To date, studies have explored the opinions of students and/or professionals in the fields of medicine,⁵ nursing,⁶ speech therapy,⁷ and psychology,⁸ yet little is known about this assessment method in physiotherapy.⁹ Therefore, the objective of the current study was to assess the perceptions of physiotherapy students regarding the implementation of OSCE during supervised internships in cardiology.

Methods

This research was approved by the Research Ethics Committee of the University of Western São Paulo (Unoeste), under protocol number 62181722.9.0000. 5515. This is a descriptive, quantitative, observational, and cross-sectional study. A convenience sample of 51 students participated, who received instructions on completing the opinion checklist after the OSCE. All undergraduate physiotherapy students from Unoeste who underwent the OSCE during supervised intern-ships in cardiology from April 2021 to March 2023 were included in the study.

The examination took place at the Skills and Simulation Laboratory (Unoeste), where all students individually faced three distinct clinical scenarios (each room had one clinical scenario), each lasting 15 minutes, during the aforementioned period. In both scenarios, students entered a room with a simulation actor, and responsible professors had access to the audio from the location (Figure 1).

Figure 1
Exterior view of the Skills and Simulation Laboratory (LHABSIM) at the Universidade do Oeste Paulista; and second scenario, where the actor simulates a hospitalized patient in a cardiology ward and the student is providing care under the supervision of the responsible professor.

Students were instructed to enter and exit the rooms on hearing a bell. The first scenario simulated the initial assessment of a cardiac patient in an outpatient setting, requiring history taking, analysis of complementary exams, setting objectives, and explaining therapy sessions, along with types of exercises and guidance for the condition. The second scenario presented a clinical situation of a hospitalized patient in a cardiology ward, where the student performed a physical assessment and formulated treatment proposals for the case. Finally, in the last scenario, the student simulated a home visit performed by a physiotherapist from the Family Health Support Team (NASF in Portuguese), conducting a comprehensive assessment and outlining objectives, possible interventions, and a physiotherapeutic approach for the case. After completing all scenarios, students gathered in a classroom to discuss their overall performance and then filled out the OSCE Clinical Skills Assessment Questionnaire, which was developed by the responsible professors, based on Troncon et al.,¹ that evaluates seven aspects: A) General exam organization; B) Clinical vignettes and instructions; C) Adequacy of the tasks required; D) Quality of post-examination explanations; E) Complexity of the content involved, where students could rate each item from 1 to 5 (1 = insufficient; 2 = fair; 3 = good; 4 = great; 5 = excellent); F) Difficulty with time management; and G) Emotional stress. Items F and G were rated on a scale from 1 to 5: very low, low, moderate, high, and very high, respectively. At the end of the questionnaire, students were able to add their positive and negative perceptions and opinions about their exam experience.

Quantitative and qualitative data were organized using Excel spreadsheets. Quantitative results are presented in tables and descriptive statistics were used with absolute numbers, frequencies, and percentages for discrete numeric data. Continuous data are expressed as means and standard deviations.

Results

In total, 51 physiotherapy students responded to the Clinical Skills Assessment Questionnaire for OSCE. None of the students rated items A to E as insufficient, with over 75% rating them as great and excellent. The highest-rated aspect was item D (quality of post-examination explanations), with 100% of students assigning grades 4 and 5 (great and excellent, respectively). The lowest-rated aspect was item B (clinical vignettes and instructions), with 23.5% of students awarding grades 2 (fair) and 3 (good). Ratings for items F and G ranged from 1 to 3 (moderate, high, and very high), with 54.9% of students reporting difficulty with time management and 94.2% indicating some level of emotional stress (Table 1).

Table 1
Responses regarding the perceptions of physiotherapy students (n=51) during the Objective Structured Clinical Examination (OSCE)

Items	Insufficient	Fair	Good	Great	Excellent
A) General exam organization	0 (0.0)	0 (0.0)	2 (4.0)	12 (23.5)	37 (72.5)
B) Clinical vignettes and instructions	0 (0.0)	1 (1.9)	11 (21.6)	17 (33.4)	22 (43.1)
C) Adequacy of the tasks required	0 (0.0)	1 (1.9)	6 (11.8)	17 (33.4)	27 (52.9)
D) Quality of post-examination explanations	0 (0.0)	0 (0.0)	0 (0.0)	6 (11.8)	45 (88.2)
E) Complexity of the content involved	0 (0.0)	1 (1.9)	6 (11.8)	16 (31.4)	28 (54.9)
Items	Very low	Low	Moderate	High	Very high
F) Difficulty with time management	9 (17.6)	14 (27.5)	16 (31.4)	5 (9.8)	7 (13.7)
G) Emotional stress	1 (1.9)	2 (3.9)	10 (19.6)	14 (27.5)	24 (47.1)

• Note: values expressed in number of responses (percentage).

Students' perceptions and opinions were summarized based on positive and negative aspects of the exam, as detailed in Table 2.

Table 2
Perceptions and positive and negative opinions of physiotherapy students (n = 51) during the Objective Structured Clinical Examination (OSCE)

Positive opinions
Great assessment to identify our difficulties and thus become more attentive to patient care.
Valid method that should have a simulated test before the official OSCE.
A milestone in my journey; it allowed me to observe issues related to patient care and time management.
I consider this evaluation method very effective because it allows us to apply our acquired knowledge in practice.
Innovative approach; there could be pre-OSCE exams for all subjects to help us perform better.
Initially causes distress and anxiety, but it allows us to assess ourselves in various scenarios and identify our strengths and weaknesses.
Despite the stress, it is a unique experience.
Negative opinions
Stress generated by not knowing how it works.
OSCE does not provide real understanding; stress factor hinders.
For people with anxiety crises, it is not a good way to assess performance.
Useful, but it makes you anxious; fear of being defined by a single encounter.

The highest average score was for item D (quality of post-examination explanations), which received 4.50 ± 0.54, and the lowest average was for item F (difficulty with time management), which received 1.80 ± 0.11, as shown in Table 3.

Table 3
Perceptions of physiotherapy students (n = 51) during the Objective Structured Clinical Examination (OSCE)

Items	Mean ± SD
A) General exam organization	33 ± 0.35
B) Clinical vignettes and instructions	33 ± 0.10
C) Adequacy of the tasks required	33 ± 0.20
D) Quality of post-examination explanations	50 ± 0.54
E) Complexity of the content involved	33 ± 0.21
F) Difficulty with time management	18 ± 0.11
G) Emotional stress	31 ± 0.14

• Note: SD = standard deviation.

Discussion

The OSCE assessment method provides participant feedback based on the overall score from the clinical skills assessment questionnaire. The main finding of this study is that undergraduate physiotherapy students had a positive perception of the OSCE, highlighting the utility of the method in enriching academic training and gaining clinical practice experience. Al-Mously et al.¹⁰ evaluated medical students' experience with the OSCE and found favorable acceptance among participants.

Regarding the organization of the exam and the adequacy of the complexity level of the contents involved in this study, there was a high degree of satisfaction among participants, achieving percentages of 96% and 86.3%, respectively. On the other hand, according to Zamanzadeh et al.,¹¹ who gathered opinions from nursing teachers and students, there are several challenges associated with the organization and implementation of the OSCE, particularly concerning the infra-structure, including the resources available in the exam environment, and the lack of clear protocols and instructions for the exam. Therefore, better strategies for organizing and planning the complexity of this method should be developed to enable its implementation in more universities worldwide.¹¹

In a systematic review involving fourteen full articles, which encompassed the perception of nursing students who underwent the OSCE compared to a traditional clinical exam, it was found that the OSCE provides a more credible assessment format for evaluating students' clinical competencies.¹² Similarly, in the present study, 86.3% of participants judged the adequacy of the tasks required by the OSCE as excellent, highlighting the importance of training skills to enhance academic experience. Furthermore, regarding the clinical vignettes chosen by the professors, 76.5% of the students rated the content as excellent to outstanding, which effectively connects theory with clinical practice. In the study by Prislin et al.,¹³ which included a sample of 335 students, both students and faculty agreed that the OSCE experience reflects the skills that students need in the execution of clinical practice, thereby enhancing their professional journey. In addition, regarding time management when moving from one scenario to another, 54.9% experienced difficulty in controlling time. However, in the study of Nasir et al.,¹⁴ which included a sample of 151 students, 106 (70.2%) considered the allotted time for each scenario as sufficient.

In the study of Porto et al.,¹⁵ the anxiety level of students was also assessed before and after the exam, and a moderate percentage was obtained. In the cur-rent study, however, 94.2% of participants reported experiencing moderate to very high emotional stress caused by the exam, exacerbated by the presence of an observing professor outside, which heightened anxiety by knowing they were being evaluated. Additionally, Martínez-Pascual et al.¹⁶ evaluated the heart rate variability in physiotherapy students and demonstrated an autonomic response to stress that varied depending on the nature and content of the scenario presented in the OSCE, regardless of its individual order of appearance.

The hypothesis of this study, that there would be positive acceptance by students in conducting the exam, was confirmed. As supported in the literature, the OSCE is an innovative method in the field of physiotherapy, encompassing a new assessment format for students that allows for analysis based on their clinical experience in an academic setting and enhances their professional development.

As a limitation of this study, the need for further research involving the application of this method in more Brazilian physiotherapy universities is pointed out, as significant findings are primarily located internationally and in other health areas.

Conclusion

The OSCE is an additional assessment tool for physiotherapy students and professors, providing an opportunity to practice patient care and acquire knowledge.

Regarding students' perceptions, the most important aspect is the feedback provided by the professors post-exam, which evaluates how they perform in specific patient care scenarios and provides guidance areas for improvement.

References

• ¹
  Troncon LE. Clinical skills assessment: limitations to the introduction of an "OSCE" (Objective Structured Clinical Exami-nation) in a traditional Brazilian medical school. Sao Paulo Med J. 2004;122(1):12-7. DOI https://doi.org/10.1590/s1516-31802004000100004
  » https://doi.org/10.1590/s1516-31802004000100004
• ²
  Alkhateeb N, Salih AM, Shabila N, Al-Dabbagh A. Objective structured clinical examination: Challenges and opportunities from students' perspective. PLoS One. 2022;17(9):e0274055. DOI https://doi.org/10.1371/journal.pone.0274055
  » https://doi.org/10.1371/journal.pone.0274055
• ³
  Henry A, Dormegnie LP, Moura TF, Olory-Garnotel L, Ramoul CE, Maffre C, et al. Mise en place des examens cliniques objectifs structurés (ECOS) facultaires en ophtalmologie. J Fr Ophtalmol. 2022;45(7):803-11. DOI https://doi.org/10.1016/j.jfo.2022.02.018
  » https://doi.org/10.1016/j.jfo.2022.02.018
• ⁴
  Yu J, Lee S, Kim M, Lee J. Comparison of students' perfor-mance of objective structured clinical examination during clinical practice. Korean J Med Educ. 2020;32(3):231-5. DOI https://doi.org/10.3946/kjme.2020.168
  » https://doi.org/10.3946/kjme.2020.168
• ⁵
  Brown C, Morse J, Nesvadba D, Meldrum A. Twelve tips for introducing simulation-based assessment in the objective structured clinical examination. Med Teach. 2021;43(4):380-3. DOI https://doi.org/10.1080/0142159x.2020.1789084
  » https://doi.org/10.1080/0142159x.2020.1789084
• ⁶
  Quigley D, Regan J. Introduction of the Objective Structured Clinical Examination in speech and language therapy education: student perspectives. Folia Phoniatr Logop. 2021;73(4): 316-25. DOI https://doi.org/10.1159/000508445
  » https://doi.org/10.1159/000508445
• ⁷
  Chen SH, Chen SC, Lai YP, Chen PH, Yeh KY. The objective structured clinical examination as an assessment strategy for clinical competence in novice nursing practitioners in Taiwan. BMC Nurs. 2021;20(1):91. DOI https://doi.org/10.1186/s12912-021-00608-0
  » https://doi.org/10.1186/s12912-021-00608-0
• ⁸
  Bellido-Esteban A, Beltrán-Velasco AI, Ruisoto-Palomera P, Nikolaidis PT, Knechtle B, Clemente-Suárez VJ. The effect of Psychology Objective Structured Clinical Examination scenarios presentation order on students autonomic stress response. Front Psychol. 2021; 12:622102. DOI https://doi.org/10.3389/fpsyg.2021.622102
  » https://doi.org/10.3389/fpsyg.2021.622102
• ⁹
  Martínez-Pascual B, Ramírez-Adrados A, Fernández-Martínez S, Gonzalez-de-Ramos C, Fernández-Elías VE, Clemente-Suárez VJ. Autonomic stress response of physiotherapy student in the different scenarios of an objective structured clinical examination. BMC Med Educ. 2022;22(1):811. DOI https://doi.org/10.1186/s12909-022-03903-9
  » https://doi.org/10.1186/s12909-022-03903-9
• ¹⁰
  Al-Mously N, Nabil NM, Salem R. Student feedback on OSCE: an experience of a new medical school in Saudi Arabia. Med Sci Educ. 2012;22:10-6. Link https://link.springer.com/article/10.1007/BF03341746
  » https://link.springer.com/article/10.1007/BF03341746
• ¹¹
  Zamanzadeh V, Ghaffari R, Valizadeh L, Karimi-Moonaghi H, Johnston ANB, Alizadeh S. Challenges of objective structured clinical examination in undergraduate nursing curriculum: Experiences of faculties and students. Nurse Educ Today. 2021; 103:104960. DOI https://doi.org/10.1016/j.nedt.2021.104960
  » https://doi.org/10.1016/j.nedt.2021.104960
• ¹²
  Vincent SC, Arulappan J, Amirtharaj A, Matua GA, Al Hashmi I. Objective structured clinical examination vs traditional clinical examination to evaluate students' clinical competence: A systematic review of nursing faculty and students' perceptions and experiences. Nurse Educ Today. 2022;108: 105170. DOI https://doi.org/10.1016/j.nedt.2021.105170
  » https://doi.org/10.1016/j.nedt.2021.105170
• ¹³
  Prislin MD, Fitzpatrick CF, Lie D, Giglio M, Radecki S, Lewis E. Use of an objective structured clinical examination in evaluating student performance. Fam Med. 1998;30(5):338-44. Link https://europepmc.org/article/med/9597531
  » https://europepmc.org/article/med/9597531
• ¹⁴
  Nasir AA, Yusuf AS, Abdur-Rahman LO, Babalola OM, Adeyeye AA, Popoola AA, et al. Medical students' perception of objective structured clinical examination: a feedback for process improvement. J Surg Educ. 2014;71(5):701-6. DOI https://doi.org/10.1016/j.jsurg.2014.02.010
  » https://doi.org/10.1016/j.jsurg.2014.02.010
• ¹⁵
  Porto, FR, Ribeiro MA, Ferreira LA, Oliveira RG, Devito KL. In-person and virtual assessment of oral radiology skills and competences by the Objective Structured Clinical Examination. J Dent Educ. 2023;87(4):505-13. DOI https://doi.org/10.1002/jdd.13138
  » https://doi.org/10.1002/jdd.13138
• ¹⁶
  Martínez-Pascual B, Ramírez-Adrados A, Fernández-Martínez S, Gonzalez-de-Ramos C, Fernández-Elías VE, Clemente-Suárez VJ. Autonomic stress response of physiotherapy student in the different scenarios of an objective structured clinical examination. BMC Med Educ. 2022;22(1):811. DOI https://doi.org/10.1186/s12909-022-03903-9
  » https://doi.org/10.1186/s12909-022-03903-9

Edited by

Associate editor: Ana Paula Cunha Loureiro

Publication Dates

• Publication in this collection
  02 Sept 2024
• Date of issue
  2024

History

• Received
  07 Feb 2024
• Reviewed
  22 May 2024
• Accepted
  18 July 2024