Competency matrix related to Covid-19: contributions from students and residents

Abstract:

Introduction:   Competency-based education has been discussed under the new perspective of the COVID-19 pandemic. The need for social distancing had effects on student activities and educational institutions needed to reflect and redesign the teaching-learning process. It was necessary to incorporate contents related to COVID-19 into the medical training programs and to adapt the teaching tools. Therefore, it is necessary that training be mediated by a collective construction of the competency matrix with the participation of the subjects involved in the process.

Objective:   This project aims to present the reconstruction of the curriculum in cardiology by observing the collaboration of the students and residents.

Method:   Exploratory study, involving 13 undergraduate medical students and eight medical residents from the institution’s cardiology program, who comprised the “Students and residents’ panel”. Consensus was reached among the panelists through the Delphi methodology. The first version of the matrix was prepared by the research team and sent, together with the FICF, containing 16 items aimed at COVID-19 content to be achieved at the cardiology internship and residency. The participants attributed their degree of agreement for each proposed item and after returning them, the data were tabulated, stored in an Excel spreadsheet and percentages for each item were calculated and presented in a descriptive manner.

Result:   The panel consisted of 19 participants. Of the 21 invited, 02 participants did not answer the online questionnaire and were excluded. The initial version of the matrix reached a consensus in the first round, with the lowest agreement rate being 71% in the internship matrix and 89.5% in the residency matrix. Both matrices showed high levels of agreement. There were no disagreements or suggestions for new items for the matrix.

Conclusion:   The students and residents’ engagement in the pedagogical process may contribute to a better understanding of the competencies for their training and bring sustainable changes to the curriculum.

Keywords: Competency-Based Education; Curriculum; Medical Education; Covid-19

Resumo:

Introdução:   A educação baseada em competências vem sendo discutida sob a nova perspectiva da pandemia da Covid-19. A necessidade do distanciamento social trouxe repercussão nas atividades estudantis, e as instituições de ensino precisaram refletir e redesenhar o processo de ensino-aprendizagem. Houve a necessidade de inserção de conteúdos relacionados à Covid-19 nos programas de capacitação médica e a adaptação das ferramentas pedagógicas. Portanto, é necessário que a formação seja mediada por uma construção coletiva da matriz de competências com participação dos sujeitos envolvidos no processo.

Objetivo:   Este projeto visa apresentar a reconstrução da matriz de competências em cardiologia observando a colaboração do estudante e residente.

Método:   Trata-se de um estudo exploratório envolvendo 13 graduandos do curso de Medicina e oito residentes médicos do programa de cardiologia da instituição, os quais compuseram o “Painel de estudantes e residentes”. Obteve-se o consenso entre os painelistas por meio da metodologia Delphi. A primeira versão da matriz foi elaborada pela equipe de pesquisa e enviada, com o TCLE, contendo 16 itens voltados para conteúdos sobre a Covid-19 a serem alcançados no internato e na residência em cardiologia. Os participantes atribuíram o grau de concordância a cada item proposto, sendo os dados apresentados de forma descritiva e com proporção de concordância.

Resultados:   O painel final foi formado por 19 integrantes. Dentre os 21 convidados, dois foram excluídos por não responderem ao questionário. A versão inicial da matriz obteve consenso na primeira rodada, sendo a menor taxa de concordância de 71% na matriz do internato e 89,5% na matriz da residência. Ambas as matrizes apresentaram índices elevados de concordância. Não houve discordâncias nem sugestões de novos itens para a matriz.

Conclusão:   O engajamento de estudantes e residentes no processo pedagógico pode contribuir para melhor compreensão sobre as competências para sua formação e promover mudanças sustentáveis no currículo.

Palavras-chave: Educação Baseada em Competência; Currículo; Educação Médica; Covid-19

INTRODUCTION

Competency-based education has been studied and debated around the world, due to the growing concern about excellence in medical training that is adequate to the needs of the population^1),(2. The discussion around this type of training occurs at the undergraduate and postgraduate levels and has gained a new focus with the emergence of the COVID-19 pandemic³.

The new coronavirus pandemic has imposed social distancing and the consequent reduction and/or cessation of educational activities, a fact that forced educational institutions to rethink the entire teaching-learning process^4),(5. This fact is even more relevant when considering medical education, whether in the context of undergraduate school or in the medical residency program of the different specialties, whose graduates had their work directed to the frontline of care for the population with COVID-19. Therefore, the already existing movements supporting more targeted training have been growing and, to a certain extent, accelerated, either in terms of the volume of experience to be had, or the time necessary for the acquisition of skills³.

The inclusion of specific content related to COVID-19 became mandatory in the several medical training programs, as well as the adjustment of pedagogical tools to carry out a greater number of activities in a not-in-person format. Although there are many national and global entities involved in the discussion on competency-based teaching^6),(7, there is no consensus on an ideal model, particularly when thinking about teaching “in times of COVID-19”.

Initiatives to improve teaching/learning are also taking place at Hospital Universitário Onofre Lopes/Empresa Brasileira de Serviços Hospitalares (HUOL/EBSERH) - Universidade Federal do Rio Grande do Norte (UFRN), where the Medical Residency Program (MRP) was created in 1980, currently receiving more than 100 resident physicians each year, distributed into 24 programs in the different specialties. This concern was the subject of research by Pita in 2018, whose Master’s Degree dissertation in health education showed a competency matrix for medical residency in Cardiology. From an in-depth analysis, there was a consensus among the panelists on the essential contents for the training in cardiology⁸.

In this context, considering the relevance of the determination of essential competencies, influenced by local, political, social, and economic factors and with the emergence of a pandemic, it is necessary to consolidate the training mediated by the collective construction of a competency matrix that, in theory, should not be simply extrapolated from pre-existing models. It should be built respecting local particularities, with the participation of all subjects involved in the process, including preceptors/teachers and residents/students. Thus, the product arising from this construction becomes more suitable for each institution⁷.

The present project intends to show the adequacy to the competency matrix in cardiology due to the COVID-19 pandemic, considering the contribution of the students and residents in cardiology for the consensus process to obtain it.

METHODS

This is an exploratory study, involving undergraduate medical students and medical residents of the institution’s cardiology program. The modified Delphi methodology was used to achieve consensus among the participants.

The Delphi method is used to seek a convergence of opinions from a group of experts, called “the panel”, when there is scarce data in the literature on a given subject or if they cannot be safely projected into the future or when seeking to stimulate the emergence of new ideas^9)-(11.

This is based on the exchange of information from responses to an interactive questionnaire, which circulates through several rounds among the panel members until a satisfactory level of convergence is achieved and, consequently, the group’s consensus. Therefore, it is assumed that the collective contribution in a structured way results in a more robust product than the one resulting from individual participation^9)-(11.

Thirteen undergraduate students working with teaching and tutoring projects, whose topics involve pedagogical and curricular development related to the teaching of cardiology and/or urgency and emergency medicine, were invited to be part of the discussion panel, in addition to the eight residents of the medical residency program in cardiology of the institution.

The option of creating a panel consisting of undergraduate and medical residency students places these individuals as protagonists in the teaching-learning process, allowing them to effectively contribute to the improvement of the curriculum structure.

The absence of a response to the questionnaire, at any round, was considered as an exclusion criterion. This panel was called the “Students and residents’ panel”.

The initial version of the competency matrix with the essential contents related to COVID-19 that should integrate the MRP competency matrix in Cardiology and the internship was created by the research team (researcher and advisor), comprising the semi-structured online questionnaire sent to participants.

This first version of the matrix was sent by email to the 21 students and residents who constituted the panel of students and residents. Therefore, each individual was able to attribute the degree of agreement to each proposed item according to the five-point Likert scale (I strongly disagree, I disagree, I neither agree nor disagree, I agree, and I strongly agree). At the time when the questionnaire was sent, instructions were given on how to complete and return it, as well as succinct and objective information on the Delphi methodology, in addition to the Free and Informed Consent Form (FICF), which contained a detailed explanation about the project.

After the feedback from the panelists and the analysis of the data collected from the initial proposed version, the first matrix version was obtained. This same process could be restarted, generating versions and subsequent rounds of consensuses, until the construction of the final version, achieved when the matrix items reached, at least, 50% of agreement between the panelists.

The data were tabulated, respecting the anonymity, and the percentages of agreement were calculated for each item. The sum of the percentages included in the options “I agree” and “I totally agree” was considered as a 50% convergence. After reaching a minimum agreement of 50%, the matrix was made available to start the teaching strategies aiming to achieve consensual competencies.

The study was approved by the Research Ethics Committee (CEP-HUOL) under CAEE number 2412316.4.0000.5292.

RESULTS

Of the 21 invited participants, two were excluded for not answering the questionnaire. Therefore, the consensus was obtained with a panel of 19 students (87.7%), of which 13 were monitors (100%) and six residents (75%), as shown in Chart 1. The mean age was 24.5±3.3 years old, with a predominance of women (52%).

Chart 1
Distribution of participants among students, monitors of monitoring projects related to Cardiology, and residents in Cardiology at HUOL/UFRN/EBSERH. HBUEA: Basic Skills in Urgency and Emergency in Adults (Habilidades Básicas em Urgência e Emergência no Adulto).

The initial matrix version consisted of 16 items focused on the essential contents on COVID-19 to be achieved at internship and medical residency programs in cardiology. The consensus was achieved at the first round, with the lowest agreement rate being 71%, observed for the item “To contribute to the development and/or updating of protocols aimed at individuals with heart disease with COVID”, in the internship matrix.

When analyzing the matrix for the residency in cardiology, the lowest rate of agreement was observed for the item “To develop reference and counter-reference for suspected COVID cases”, which reached 89.5% of agreement.

There were no divergences, or suggestions for new items for the matrix composition.

The agreement rates are shown in Frames 1 and 2.

Frame 1
Rate of agreement of the students and residents’ panel considering training during the medical school internship.

Contents/Competency Items	Agreement (%)
	1	2	3	4	5
To know the epidemiological, social, and pathophysiological aspects of the new coronavirus.	0	0	0	52.6	47.4
To identify the clinical characteristics inherent to COVID 19, recognizing the susceptibility of different populations	0	0	0	21	79
To identify the clinical signs and symptoms of COVID 19.	0	0	0	10.5	89.5
To develop a differential diagnosis, considering the respiratory distress syndrome.	0	0	0	21	79
To rationally use the diagnostic and therapeutic arsenal available to individuals with COVID-19.	0	0	0	31.6	68.4
To develop reference and counter-reference for suspected COVID-19 cases.	0	0	0	52.6	47.4
To demonstrate the correct use of PPE	0	0	0	15.8	84.2
To demonstrate the correct removal of PPE to care for suspected and confirmed individuals of COVID-19.	0	0	0	15.8	84.2
To contribute to the development and/or updating of protocols aimed at individuals with heart disease and COVID-19.	0	0	21	47.4	31.6
To adequately record the information in medical files.	0	0	0	36.8	63.2
To assist the individual with COVID-19 in urgency and emergency situations, such as CPR	0	0	0	21	79
To know and demonstrate the correct use of mechanical ventilation in individuals with COVID-19.	0	0	0	31.6	68.4
To know and use digital health tools such as telehealth and teleconsultation.	0	0	5,3	57.9	36.8
To communicate difficult news related to COVID-19, while also promoting the population’s education.	0	0	0	42.1	57.9
To work collaboratively, respecting the particularities of other team professionals.	0	0	0	36.8	63.2
To understand the importance of self-care.	0	0	0	42.1	57.9

• 1 = I totally disagree; 2 = I disagree; 3 = I neither agree nor disagree; 4 = I agree; 5 = I strongly agree.

Frame 2
Rate of agreement of the students and residents’ panel considering training during medical residency in cardiology.

Contents/Competency Items	Agreement (%)
	1	2	3	4	5
To know the epidemiological, social, and pathophysiological aspects of the new coronavirus.	0	0	0	42,1	57,9
To identify the clinical characteristics inherent to COVID-19, recognizing the susceptibility of different populations	0	0	0	26,3	73,7
To recognize the clinical signs and symptoms of COVID-19.	0	0	0	21	79
To develop a differential diagnosis considering the respiratory distress syndrome.	0	0	0	15,8	84,2
To rationally use the diagnostic and therapeutic arsenal available to individuals with COVID, based on the best scientific evidence.	0	0	0	21	79
To develop reference and counter-reference for suspected cases of COVID-19.	0	0	0	15,8	73,7
To demonstrate the correct use of PPE.	0	0	5,2	15,8	79
To demonstrate the correct removal of PPE to care for suspected and confirmed individuals of COVID-19.	0	0	5,2	15,8	79
To contribute to the development and/or updating of protocols aimed at individuals with heart disease with COVID-19.	0	0	0	31,6	68,4
To adequately record the information in medical files.	0	0	0	21	79
To assist the individual with COVID-19 in urgency and emergency situations, such as CPR	0	0	0	15,8	84,2
To know and demonstrate the correct use of mechanical ventilation in individuals with COVID-19.	0	0	0	21	79
To know and use digital health tools such as telehealth and teleconsultation.	0	0	0	47,4	53,6
To communicate difficult news related to COVID-19, while also promoting the population’s education.	0	0	0	26,3	73,7
To work collaboratively, respecting the particularities of other team professionals.	0	0	0	31,6	68,4
To understand the importance of self-care.	0	0	0	26,3	73,7

• 1 = I totally disagree; 2 = I disagree; 3 = I neither agree nor disagree; 4 = I agree; 5 = I strongly agree.

DISCUSSION

The consensus regarding the competencies related to COVID-19 was quickly reached and showed high levels of agreement in all items of the matrix. This high rate of agreement and the fact that a consensus was attained fast, as early as in the first round, can be explained by the use of the Delphi¹² strategy, which may have been influenced by the panelists’ (students and residents) level of knowledge on the subject, since the COVID-19 topic has required constant studies and urgent decisions.

The item “To contribute to the development and/or updating of protocols aimed at individuals with heart disease and COVID-19” showed the lowest percentage of agreement (79%), with this being related to training during medical school internship. This fact can perhaps be explained by the training model, frequently based on the traditional teaching model, where the teacher is the holder of knowledge¹³ and, consequently, the student assumes a more passive role, contributing little to actions such as the one addressed in the item.

On the other hand, initiatives that encourage active and collaborative student participation are important for a more robust and active learning, favoring their engagement in the process of health education, permanent education, and the creation/development of improvement strategies both in the area of health and in education. Especially at a time when actions are needed for remote emergency education due to the pandemic, it is worth emphasizing the relevance of the inclusion of the competency matrix¹⁴, not only of contents that meet current demands¹⁵, but also of teaching and evaluation strategies that allow student engagement and proactivity^16)-(18.

The items related to health care were the ones with the highest percentage of strong agreement (I totally agree), such as “To demonstrate the correct use of PPEs” and “To demonstrate the correct removal of PPE to care for suspected and confirmed individuals of COVID-19” for the internship training. As for the residency training, the items “To provide assistance to individuals with COVID-19 in urgency and emergency situations such as CPR” and “To develop a differential diagnosis considering the respiratory distress syndrome” were the ones with strong agreement, drawing attention to the urgency assistance itself and to the need and/or difficulty in preparing the differential diagnoses imposed by COVID-19.

The results of the research showed that the proportion of agreement was quite similar, considering the level of education of the respondents, whether undergraduate students or residents, and reinforce the importance of the contribution of students and residents in-training in the development of curricular matrices, as already described in the literature¹⁹. There was a clear concern about the scientific evidence support, including the ability to interpret scientific articles and perform critical judgment and adapt diagnostic and therapeutic approaches, as well as emphasizing the importance of developing the ability to share knowledge about COVID- 19 with patients and the community in a clear, objective, and accessible way. This adequacy corroborates the suggestion of the Accreditation Council for Graduate Medical Education (ACGME)⁷ as essential competencies in undergraduate training, which are extrapolated to the scope of medical residencies, including clinical reasoning, adequate information management, interpersonal and team communication skills and professionalism. Finally, it is essential to emphasize the important role of digital technologies throughout the teaching-learning process, from action planning to evaluation, in different scenarios and for different populations, with the aim of maximizing learning and when it is crucial to use the remote modality, such as during the pandemic^20),(21.

CONCLUSIONS

When invited to participate in the pedagogical process, students and residents quickly respond to the call, bringing relevant contributions to the teaching-learning process. Getting students involved in this process and encouraging them to understand the attributes and competencies relevant to their training, even at an unstable and rapidly changing scenario, can represent an extremely robust pedagogical strategy, promoting sustainable changes in the curriculum.

ACKNOWLEDGMENTS

To the students and residents who contributed to the study and to the Professional Master’s Degree in Health Education at Universidade Federal do Rio Grande do Norte (MPES/UFRN).

REFERENCES

• ¹ Gontijo ED, Alvin C, Megale L, Melo JRC, Lima MECC. Matriz de competências essenciais na formação médica. Rev Bras Educ Med. 2013;37(4):526-39.
• ² Aguiar AC, Ribeiro ECO. Conceito e avaliação de habilidades e competência na educação médica: percepções atuais dos especialistas. Rev Bras Educ Med . 2010;34(3):371-8.
• ³ Goldhamer MEJ, Pusic MV, Co JPT, Weinstein DF. Can Covid catalyze an educational transformation? Competency-based advancement in a crisis. N Engl J Med, 2020;383:1003-5. [access in 10 aug 2020]. Available from: Available from: https://www.nejm.org/doi/full/10.1056/NEJMp2018570
  » https://www.nejm.org/doi/full/10.1056/NEJMp2018570
• ⁴ World Health Organization. WHO announces COVID-19 outbreak a pandemic. WHO; 2020 [access in 11 aug 2020]. Available from: Available from: https://www.euro.who.int/en/health-topics/health-emergencies/coronavirus-covid-19/news/news/2020/3/who-announces-covid-19-outbreak-a-pandemic%0A
  » https://www.euro.who.int/en/health-topics/health-emergencies/coronavirus-covid-19/news/news/2020/3/who-announces-covid-19-outbreak-a-pandemic%0A
• ⁵ World Health Organization. Infection prevention and control for the safe management of a dead body in the context of COVID-19. J Hosp Infect. 2020;104(3):246-51 [access in 10 aug 2020]. Available from: Available from: https://www.who.int/publications/i/item/infection-prevention-and-control-for-the-safe-management-of-a-dead-body-in-the-context-of-covid-19-interim-guidance
  » https://www.who.int/publications/i/item/infection-prevention-and-control-for-the-safe-management-of-a-dead-body-in-the-context-of-covid-19-interim-guidance
• ⁶ Fernandes CR, Farias Filho A, Gomes JMA, Pinto Filho WA, Cunha GKF, Maia FL. Currículo baseado em competências na residência médica. Rev Bras Educ Med . 2012;36(1):129-36.
• ⁷ Accreditation Council for Graduate Medical Education. Clinical Competency Committees [access in 19 mar 2018]. Available from: Available from: http://www.acgme.org/
  » http://www.acgme.org/
• ⁸ Pita CG. Matriz de competência para residência médica em cardiologia da Universidade Federal do Rio Grande do Norte [dissertação]. Natal: Universidade Federal do Rio Grande do Norte; 2018 [access in 10 aug 2020]. Available from: Available from: https://repositorio.ufrn.br/jspui/handle/123456789/25719
  » https://repositorio.ufrn.br/jspui/handle/123456789/25719
• ⁹ Massaroli A, Martini JG, Lino MM, Spenassato D, Massaroli R. Método Delphi como referencial metodológico para a pesquisa em enfermagem. Texto Context - Enferm. 2017;26(4):1-9.
• ¹⁰ Wright JTC, Giovinazzo RA. Delphi: uma ferramenta de apoio ao planejamento prospectivo. Cad Pesq Adm. 2000;1(12):54-65.
• ¹¹ Silva RF, Tanaka OY. Técnica Delphi: identificando as competências gerais do médico e do enfermeiro que atuam em atenção primária de saúde. Rev Esc Enferm USP. 1999;33(3):207-16.
• ¹² Marques JBV, Freitas D. Método DELPHI: caracterização e potencialidades na pesquisa em Educação. Pro-Posições. 2018;29(2):389-415.
• ¹³ Harden RM, Crosby J. AMEE Guide nº 20: The good teacher is more than a lecturer - the twelve roles of the teacher. Med Teach. 2000;4(22):334-57.
• ¹⁴ Ashokka B, Ong SY, Tay KH, Loh NHW, Gee CF, Samarasekera DD. Coordinated responses of academic medical centres to pandemics: sustaining medical education during COVID-19. Med Teach. 2020;42(7):762-771.
• ¹⁵ Epstein MR, Hundert AM. Defining and assessing professional competence. JAMA. 2002;287(2):226-35.
• ¹⁶ Lu CY, Nguyen Q, Ersin OH. Active student engagement in curriculum development. Am J Pharm Educ. 2015;79(2):30.
• ¹⁷ Jenkins D. Teaching leadership online: an exploratory study of instructional and assessment strategy use. J Leadersh Educ. 2014;15(2):129-49.
• ¹⁸ Stefani L. Designing the curriculum for student engagement. All Ireland Journal of Higher Education. 2009; 1(1):11.1-11.3 [access in 13 aug 2020]. Available from: Available from: https://ojs.aishe.org/index.php/aishe-j/article/view/11
  » https://ojs.aishe.org/index.php/aishe-j/article/view/11
• ¹⁹ Bovill C, Morss K, Bulley CJ. Should students participate in curriculum design? Discussion arising from a first year curriculum design project and a literature review. Pedagogic Research in Maximising Education. 2009;3(2):17-26.
• ²⁰ Ellaway R, Masters K. AMEE Guide 32: E-Learning in medical education Part 1: Learning, teaching and assessment. Med Teach . 2008;30(5):455-73. doi: 10.1080/01421590802108331.
  » https://doi.org/10.1080/01421590802108331
• ²¹ Car J, Carlstedt-Duke J, Car LT, Posadzki P, Whiting P, Zary N, et al. Digital education in health professions: the need for overarching evidence synthesis. J Med Internet Res. 2019;21(2):e12913 doi: 10.2196/12913.
  » https://doi.org/10.2196/12913

• 2
  Evaluated by double blind review process.
• FUNDING
  We declare no funding for this research.

Edited by

• Chief Editor: Rosiane Viana Zuza Diniz. Associate Editor: Roberto Zonato Esteves.

Publication Dates

• Publication in this collection
  17 May 2021
• Date of issue
  2021

History

• Received
  29 Sept 2020
• Accepted
  12 Mar 2021