Transcultural adaptation of the SETQ Smart for preceptors in BrazilABSTRACT
Introduction: The evaluation of clinical preceptors is essential to ensure the quality of resident training in medical postgraduate education. In Brazil, there is a shortage of validated instruments to assess preceptors’ competencies, especially in the clinical context.
Objective: This study aimed to perform the cross-cultural adaptation of the self-assessment version of the System for Evaluation of Teaching Qualities (SETQ) Smart for use by physician preceptors in the Brazilian context.
Method: This was a methodological and cross-sectional study of cross-cultural adaptation, conducted according to international guidelines. The stages included: initial translation into Brazilian Portuguese, synthesis of the translated versions, back-translation, analysis by an expert committee to assess idiomatic and cultural equivalence, and pre-testing with physician preceptors. Content validity was assessed by a panel of expert judges.
Result: In the pre-test phase, 38 physician preceptors participated, of whom 57.9% were male. Except for the initial prompt, 80% of participants considered the SETQ Smart domains clear and culturally appropriate. For content validity analysis, a panel of 10 expert preceptors was formed, with a median age of 43 years (range: 38.0-46.8) and median time working as preceptors of 10.5 years (range: 5.8-15.0). Agreement among the judges exceeded 80% for all items evaluated.
Conclusion: The self-assessment version of the SETQ Smart for physician preceptors was culturally adapted for the Brazilian context. The study provided preliminary evidence of content validity, demonstrating that the instrument is understandable, appropriate and potentially useful for evaluation in clinical teaching in medical residency programs in Brazil.
Keywords:
Assessment Medical education; Medical residency; Preceptor; Self-assessment
RESUMO
Introdução: A avaliação do preceptor é fundamental para assegurar a qualidade da formação dos residentes na pós-graduação médica. No Brasil, há uma escassez de instrumentos validados que avaliem as competências dos preceptores, especialmente no contexto clínico.
Objetivo: Este estudo teve como objetivo adaptar transculturalmente para o contexto brasileiro a versão de autoavaliação do System for Evaluation of Teaching Qualities (SETQ) Smart, voltada para médicos preceptores.
Método: Trata-se de um estudo metodológico e transversal de adaptação transcultural, conduzido segundo diretrizes internacionais. As etapas incluíram: tradução inicial para o português brasileiro, síntese das versões traduzidas, retrotradução, análise por um comitê de especialistas para verificar a equivalência idiomática e cultural, e pré-teste com médicos preceptores. A validade de conteúdo foi avaliada por um painel de juízes especialistas.
Resultado: Na fase de pré-teste, participaram 38 médicos preceptores, dos quais 57,9% eram homens. Com exceção do enunciado inicial, 80% dos participantes consideraram os domínios do SETQ Smart claros e culturalmente apropriados. Para análise de validade de conteúdo, um painel de dez preceptores especialistas foi formado, com mediana de idade de 43 anos (intervalo: 38,0-46,8) e tempo de atuação como preceptor de 10,5 anos (intervalo: 5,8-15,0). A concordância entre os juízes superou 80% em todos os itens avaliados.
Conclusão: A versão de autoavaliação do SETQ Smart para médicos preceptores foi adaptada culturalmente para o contexto brasileiro. O estudo forneceu evidências preliminares de validade de conteúdo, demonstrando que a ferramenta é compreensível, adequada e potencialmente útil para avaliação no ensino clínico da residência médica no Brasil.
Palavras-chave:
Educação Médica; Residência Médica; Preceptor; Autoavaliação
INTRODUCTION
Medical residency (MR) is an in-service learning programme that requires periodic evaluations, as established by the National Medical Residency Commission (CNRM)1. Continuous assessment is fundamental not only for residents, but also for preceptors, guaranteeing quality teaching1)-(3. The CNRM is responsible for promoting continuing training and regularly evaluating the performance of preceptors with well-defined criteria, ensuring that they have access to the results of the evaluations2.
Teaching in the clinical environment presents specific challenges and is often carried out without formal pedagogical preparation. Clinical preceptors need to go beyond technical competence, developing educational skills that set them apart from teachers in traditional academic environments. Although the majority have solid training in medical knowledge, pedagogical training is still a significant gap4).(5.
Studies show that many clinical educators have limited insight into their own performance, reinforcing the need for structured assessments3. With the growing care load, the time allocated to teaching is reduced, making it essential to apply effective educational methodologies. However, the lack of clear guidelines and structured, institutionalised assessment tools compromises the accurate assessment of preceptors’ competencies4), (6)-(13.
To guarantee the quality of MR, the evaluation of preceptors must be centred on results and include multiple criteria, such as academic training, feedback and reflective self-assessment practices. Validated instruments are essential for measuring teaching performance. Although the CNRM does not prescribe specific tools, there is a growing effort to identify essential competencies for effective medical educators5. However, Brazil still faces challenges in structuring a standardised system for assessing preceptors, resulting in heterogeneous approaches4),(6),(9)-(13.
A systematic review revealed that several assessment tools lack a clear theoretical framework and fail to incorporate crucial variables17)-(20. Among the few robust systems is the System for Evaluation of Teaching Qualities (SETQ), developed in the Netherlands and based on the principles of the Canadian Medical Education Directions for Specialists (CanMEDS) (21), (22)-(26. The SETQ Smart, an improved version, combines preceptor self-assessment and feedback from residents, promoting continuous improvements in clinical teaching23. Studies indicate that repeated application of the SETQ improves preceptors’ perception of their skills and areas for improvement, making it a reliable feedback and teacher development model3),(23.
Considering that the evaluation of medical preceptors plays an essential role in guaranteeing the quality of medical residency programmes, and that studies indicate that the combination of self-assessment and feedback from residents is an effective strategy for improving teaching skills, it is essential to adopt a structured instrument that contributes to strengthening the training of these professionals. This improvement not only has a direct impact on the qualifications of future specialists, but also on improving the health care provided to the population.
However, Brazil lacks standardised national tools for evaluating preceptors, especially those that prioritise self-assessment as a central professional development strategy. Given this gap, this study proposes the cross-cultural translation and adaptation of the self-assessment version of the SETQ Smart for medical preceptors, with the aim of enabling its application in the Brazilian context. The implementation of this instrument could represent a significant advance in the qualification of preceptors, promoting a more reflective teaching practice in line with the demands of medical training in the country.
METHOD
Study design and ethical principles
This is a methodological, cross-sectional study. The research project was submitted to the Institutional Ethics Committee (CAAE:60372322.6.1001.0121) with a favourable opinion for its execution and all participants signed the informed consent form.
The Instrument
SETQ Smart is a system made up of two questionnaires, one of which is for the self-assessment of medical preceptors24)-(26. The version aimed at medical preceptors showed strong evidence of validity.
The instrument, which has 30 items and six domains, includes seven categories for clinical teaching with a Likert response scale of 7 (totally disagree - totally agree), as well as a complementary item that analyses overall performance on a 10-point scale.
Study stages and participants
The cross-cultural adaptation stage adopted the international recommendations and began by translating the SETQ Smart into the language spoken in Brazil27)-(29. The instrument in its original English version was translated by two Brazilian translators fluent in English who independently produced two versions called T1 and T2. Two authors and a third bilingual researcher met to discuss inconsistencies and relevance in order to synthesise versions T1 and T2 into a single version of the SETQ for Brazilian Portuguese called T1,2. This version was back-translated into English by two native translators from English-speaking countries, fluent in Brazilian Portuguese and without access to the original instrument. Both translators independently produced two versions of the SETQ in English called RT1 and RT2. An expert committee was organised comprising three specialists in cross-cultural adaptation research (one of whom has a Portuguese-English Language degree) and three health professionals with research experience. The aim was to compare and evaluate the following versions: original, T1, T2, RT1 and RT2 in order to discuss the disparities identified. Each member sent their thoughts on each item assessed and based on the results obtained, the researchers drew up the pre-final version. This version was pre-tested and applied to the target population, medical preceptors, to assess clarity and cultural appropriateness. The sample was selected for convenience and according to the literature adopted, between 30 and 40 participants are needed27.
The Content Validity stage was conducted with the composition of the panel of expert judges to obtain the first sources of validity evidence. The sample selection was intentional, and included Brazilian preceptors, of both sexes and linked to any medical speciality, with at least two years’ experience in preceptorship and specialisation in medical residency.
Foreign doctors were excluded from the study, as were those who were not available to answer all the questionnaire items, both for the cross-cultural adaptation stage and for Content Validity.
Data collection
Data was collected electronically using Google Forms, containing all the components for the preceptor self-assessment version. The participants evaluated and answered on a three-point Likert scale, clarity (“not at all clear”; “somewhat clear” and “entirely clear”) and cultural suitability (“not at all suitable for Brazilian culture”; “somewhat suitable for Brazilian culture” and “entirely suitable for Brazilian culture”). A new electronic form was organised, with the results obtained in the pre-test phase, to examine the Content Validity by the panel of expert judges. Participants assessed the clarity, cultural suitability and relevance of each item. At all stages of the collection, participants had an area for suggestions or comments at the end of each domain, if necessary.
Data analysis
The data was stored in Microsoft Excel® spreadsheets and analysed using the R programming language 4.1.030. Descriptive statistics were used, with measures of central tendency and variability for analysing numerical variables, and absolute and relative frequency for categorical variables31.
There is still no consensus in the literature as to the minimum value to ensure clarity and adequacy of the items included in the instrument. Therefore, it was agreed that a minimum of 80% of responses should be “entirely clear” and “entirely suitable for Brazilian culture” for this type of evaluation. The Percentage of Agreement was used to assess the degree of agreement between the experts. The calculation took into account the proportion of items that achieved an agreement rate of at least 80%. The number of these items was divided by the total number of items in the instrument and then multiplied by 100 to express the result as a percentage32.
The percentage of agreement between the judges was the parameter used to analyse content validity, also with a minimum acceptable standard of 80% agreement33.
RESULTS
Cross-cultural adaptation of the SETQ Smart
Three items relating to the speciality of anaesthesiology were adapted before proceeding to the pre-test phase. Although the SETQ tool has been validated for different specialities and clinical contexts, its enhanced version, SETQ Smart, was redesigned on the basis of a multicentre study carried out in six countries, and validated with residents and preceptors in the field of anaesthesiology. Thus, items 6 “Teach residents the full spectrum of perioperative care”, 23 “Adhere to professional practice standards in the field of anaesthesiology” and 26 “Teach residents organisational aspects of perioperative practice” were amended from the initial version in order to apply them to different clinical and surgical contexts within postgraduate medical services.
In the pre-test stage, 38 preceptors took part, with a median age of 43.0 (38.0-46.8) years, of whom 22 (57.9%) were male and 16 (42.1%) female. The median time spent as a preceptor was 10.5 (5.8-15.0) years. Of the medical preceptors interviewed, 60.3% were part of the clinical staff of the University Hospital of the Federal University of Santa Catarina (HU/UFSC) and 39.5% of the Florianópolis School of Public Health (ESP).
The areas of activity of the participating preceptors were: Cardiology (1), Head and Neck Surgery (1), Digestive System Surgery (4), Vascular Surgery (2), Internal Medicine (2), Coloproctology (1), Palliative Care (1), Endocrinology and Metabology (1), Gastroenterology (2), Paediatric Gastroenterology (1), Gynaecology and Obstetrics (2), Family and Community Medicine (15), Paediatrics (2), Psychiatry (1), Rheumatology (1).
With the exception of the wording, all the domains were considered clear and adequate by more than 80% of the judges (Figure 1).
Content Validity of the SETQ Smart
The panel of judges was made up of six (60%) females and four (40%) males, totalling 10 participants with a median age of 41.5 (34.0-48.5) years. With regard to how long the preceptors had been working, the median was 15.0 (5.8-17.3) years.
Among the participants, 70% belonged to the HU/UFSC institution and 30% to the ESP. As regards specialities, 30% were in Family and Community Medicine, 10% were Anaesthesiologists, 10% worked in General Surgery, 10% in Internal Medicine, 10% in Dermatology, 10% in Gynaecology and Obstetrics, 10% in Paediatrics and 10% in Psychiatry.
The content validity of the instrument was assessed by a panel of expert judges based on four main criteria: (1) grammatical and orthographic structure; (2) clarity and cultural suitability for the target public; (3) representativeness of the item in relation to the domain it is intended to assess; and (4) pertinence for retention in the instrument. The data obtained was organised by criterion, from the highest to the lowest percentage of agreement between the assessors (Table 1).
In general, the results indicated a high degree of consensus among the assessors, with the majority of items showing levels equal to or greater than 90.0% in all criteria. The predominance of high percentages reinforced the linguistic, cultural and conceptual suitability of the SETQ Smart. Only a small number of items (specifically in the criteria of structure, clarity and permanence) scored 80.0%, which did not compromise the overall validity, but suggests possible fine-tuning in the language or focus of certain items.
Development of the final version of the SETQ Smart self-assessment for preceptors
Based on the results obtained in the pre-test and content validity phases, the questionnaire was reformulated with the aim of generating its definitive version. The expressions “quanto a/ao” (Regarding) and “eu” (I) were added to the statements of each domain. The first domain, for example, “Learning Environment” became “Regarding the learning environment I”. The addition of this expression came about following suggestions from the expert committee in order to provide greater fluidity to the questionnaire and connect the statement to the subsequent items.
The first paragraphs at the beginning of the questionnaire introduce the content to be assessed, indicating the purpose of the tool. This topic, relating to the wording and the response scale, was considered by the preceptors to have the lowest level of clarity in the evaluation, with 28.9% of the preceptors classifying this introduction as somewhat clear, pointing to the need for more precision in the introductory text regarding (i) the objectives of the questionnaire, (ii) the way in which the domains are assessed and (iii) the measurement scale used. A rewording of the questionnaire was required and the changes are shown in Table 2.
The SETQ Smart Questionnaire was improved by using a seven-point Likert response scale, unlike the original version. This allowed for a wider range of assessments12. In our study, the statement, which also comprises the response scale, was somewhat clear to 33.2% of the participants, more precisely, the differentiation between disagree and partially disagree and between agree and partially agree, included in the response options (strongly disagree; disagree; partially disagree; neutral; partially agree; agree; strongly agree and not applicable/cannot evaluate). Since this analysis was carried out with a small number of participants, we consulted the designer of the tool about the possibility of processing the evaluations using a five-point scale in proposed future evaluations as part of subsequent validation studies. However, when investigating the variability of evaluations by residents’ who engaged the SETQ with differentiated five-point and seven-point response scales, it was found that a seven-point scale offers greater advantages34. This is because supervisors usually received very high marks in the SETQ based on the simpler five-point scale. On the other hand, the available evidence indicates that people with a high level of education prefer to have more response options when completing questionnaires. Thus, after reviewing the literature, and taking into account the study by Debets et al on the variability of residents’ evaluations of the faculty’s performance measured by five-point and seven-point response scales, the final version of the preceptors’ self-assessment was produced with the 7-point scale34. However, it is thought that in Brazil this option should be tested later in more comprehensive studies based on quantitative data.
Due to the restrictions imposed by the COVID-19 pandemic and the circumstances in which physical presence proved difficult or risky, the adoption of distance learning became a fundamental alternative to guarantee the continuity of supervision, guidance and training, even in contexts of limited personal interaction. From this experience, the use of educational technologies in this profile was consolidated, which motivated the improvement of topic 9 under “Preceptor’s professional attitude”. In this context, the original phrase “... easily accessible when on-call” has been adapted to “... easily accessible when on-call and on remote supervision duty”.
Due to the considerations put forward by the preceptors regarding the interpretation of the terms used, when developing items 19 and 20, the need arose to use examples in order to facilitate understanding of the expressions “positive feedback” and “negative feedback” in the context of performance assessment processes. As a result of this refinement, the term “positive feedback” was replaced by “I highlight desirable/appropriate behaviours already achieved”, while the term “negative feedback” was changed to “I point out professional behaviours not yet achieved and undesirable ones that need to be modified”. These changes are shown separately in Table 2.
In view of the differences over the translation of some of the original concepts, the preparation of the definitive version also required consultation with the proponents of the original version of the tool. Certain nomenclatures in English, although coherently translatable into Portuguese, have been applied with different connotations or have even ceased to be used over time in the routines of medical practice, such as the term “compaixão” (“compassion”). Accordingly, some members of the pre-test phase adopted this term with a restrictive connotation, with the sense of “feeling sorry for someone”. Moreover, this term is rarely used in medical practice, in curricula and in the usual training of group communication skills. From the answers given by the study participants, we observed a preference for the Portuguese term for “empathy” rather than “compassion”, in an alternative formulation of item 24 of the professionalism domain. However, after consulting the developer of the tool and reviewing the relevant literature, we decided to reformulate the original wording as follows: “Demonstrate empathy and integrity in my relationships with patients and families and act with compassion to alleviate patient suffering.”
We can highlight other difficulties encountered in the translation of certain terms into Portuguese. In the case of item 17 of the Assessment domain, for example, the expression “specific patients” translated into Portuguese is open to ambiguity. On the one hand, it could refer to a patient with a highly complex diagnosis or condition; and on the other, patients being assessed in specific contexts. In this case, in addition to the group of experts, we consulted two experts familiar with the medical terminology adopted in English-speaking countries to interpret the same sentence with different synonyms for the terms it contains. Thus, the initial wording of item 17 of the preceptor questionnaire “Evaluate residents’ application of knowledge to specific patients regularly” was replaced by “Evaluate the residents’ application of knowledge to each type of patient”.
The study endeavoured to improve a Brazilian version that was compatible with the original and, at the same time, maintained the originality of the questionnaire. There were changes that were limited to structural and grammatical issues, in addition to those mentioned above. After the considerations raised in the pre-test, by the Expert Judges participating in the content validation, and in consultation with the main author of the tool, the adaptations shown in Table 2 were made.
Thus, after the final version of the SETQ Smart for preceptor self-assessment was translated by a native English translator fluent in Brazilian Portuguese, the tool was sent to the main author for approval and validation. This final version of the questionnaire for resident doctors is available at: https://github.com/NaaraiCamboim/Questionario_preceptor
DISCUSSION
The results of the study indicate that although all the elements of the questionnaire were considered clear and suitable for Brazilian culture, as well as relevant to the concept under study and representative of their respective domains, there was variation in the answers. This variation was observed when compared to a previous study that used the version of the SETQ Smart focused on the residents’ perspective13. In the context of the preceptors’ self-assessment with the SETQ Smart, the considerations raised by the participants were thorough, with a detailed critical analysis of practically all of the construct’s items, resulting in a diversity of responses. This detail reflects the complexity of the preceptorship role within the postgraduate medical programme5)-(7.
In addition, the heterogeneous nature of preceptor training should be underlined, as well as the different types of training within the different medical specialities, since this study set out to cover both clinical and surgical preceptors within postgraduate medical courses, and the limitations in pedagogical competencies applied to the clinical-educational setting. While the majority of preceptors are trained in medical knowledge and clinical skills, there is a gap in training for actual teaching4)-(7. Few institutions offer ongoing support to preceptors in their role as educators, and there is a shortage of regular training and professional development programmes, which are fundamental to building a suitable learning environment for preceptors to carry out their role effectively4),(5.
The application of the SETQ in the various clinical environments, with a view to the different medical specialities, including surgical ones, was based on previous studies that validated the instrument. A multicentre study, for example, applied the original version of the SETQ to 546 residents and 629 teachers/preceptors in 29 residency programmes in different specialities, except surgery35. In another study, a multicentre prospective cohort model confirmed the reliability and validity of the SETQ for evaluations of surgical preceptors, involving 343 surgeons and 320 residents36. Similarly, a study carried out in four university hospitals in Bahrain reported the adaptation and validity of this tool for the self-assessment of preceptors in different clinical settings37. Despite the successful cross-cultural adaptation in this study, it is worth highlighting the need for a larger sample to achieve multiple sources of validity evidence, which was not possible in our study due to the short period of data collection.
The concept of empathy and compassion generated discussions in our study, reflecting discrepancies already documented in the literature38. While empathy is widely understood as the ability to put oneself in the patient’s shoes, understanding their feelings and perspectives, compassion expands this idea by including the active motivation to alleviate the other person’s suffering, translating into tangible care actions.
We observed that the study participants showed familiarity with the concept of empathy, since it is frequently used in doctor-patient communication. However, the term compassion was less understood as appropriate for Brazilian culture by some participants in the study, which suggests the need for more in-depth reflection on this topic and its importance in medical training. Considering that both empathy and compassion are essential aspects in the clinical practice and professional development of resident doctors and preceptors, we opted to reformulate item 24 of the questionnaire, incorporating both terms. This need for a more in-depth discussion of the concept was raised mainly by the preceptors, while the residents showed less emphasis on this aspect in their answers13. This finding merits further investigation to understand whether the preceptors’ arguments were limited to the definition of the term or whether there is a need to observe possible changes in compassion over time in the role.
With regard to the concept of feedback, our analyses revealed a wide variation of interpretations. The literature already points out that the understanding of feedback can differ depending on the context and the way it is applied in medical practice39. Regardless of the definitions, feedback plays a fundamental role in clinical teaching, promoting reflection and metacognition and contributing to the improvement of preceptors’ performance, provided that it’s appropriate and adapted to the specific needs of each professional and institution40.
Another noteworthy point is the scarcity of Brazilian studies that use a preceptor self-assessment tool. Some of these studies used questionnaires that had not been validated in Brazil, while others adapted their tools from the perspective of undergraduate medical students rather than resident doctors. Some other studies have been validated for a particular speciality with a specific focus on this profile of preceptors, which makes it difficult for the institution to have a more systematic institutional evaluation model to assess the general framework of standardised competencies for the entire group of preceptors. In addition, other preceptor self-assessment tools are not restricted exclusively to the preceptor’s pedagogical competencies, according to the teaching competency matrix, but cover other points related to the residency programme. This broadened scope can divert the preceptor’s attention from specific reflection on their teaching performance, compromising the assessment aimed at developing their educational skills4),(9)-(13.
The need for self-evaluation is also related to the adult learning approach. This model emphasises that adults should be responsible for their own educational decisions, actively participating in the planning and evaluation of their education. Furthermore, motivation for adult learning is more effective when based on internal factors, such as personal growth and professional development, rather than external stimuli41. By contrasting the preceptor’s perception of their teaching with that of the residents, the SETQ generates reflections on the impact of clinical teaching on daily practice and contributes to the continuous improvement of preceptorship.
Among the limitations of this study is the fact that the research was conducted in an urban centre in southern Brazil, which may limit the generalisability of the findings to other medical residency programmes in different geographical contexts and specialties. Furthermore, the relatively small number of participating preceptors restricts the robustness of the conclusions.
However, considering the lack of validated tools for preceptor self-assessment in Brazil and the evidence that clinical educators are often unclear about their own performance, this study reinforces the need to implement self-assessment practices in medical residency programmes. Previous studies have already shown that through repeated evaluation cycles, preceptors develop a better perception of their strengths and the areas that need improvement. In addition, research indicates that the implementation of SETQ has contributed to improving educational outcomes and support for preceptors3.
In light of these findings, we suggest that medical institutions establish a structured evaluation system to identify the pedagogical strengths of preceptors and the areas that need improvement. This system must be integrated into institutional practice, with regular and periodic application, to promote continuous monitoring and stimulate reflection on clinical teaching.
A multicentre study is currently underway in Brazil with the aim of further investigating the applicability of the SETQ Smart. This effort aims to validate the tool in a larger sample, covering different areas of postgraduate medical education, and to provide more robust evidence of its effectiveness in the Brazilian context. It is therefore recommended that further research be carried out to expand the applicability and validity of the SETQ Smart in the country.
FINAL CONSIDERATIONS
This study developed the Brazilian version of the SETQ Smart, with the aim of evaluating the teaching competencies of medical preceptors through self-assessment. The items contained in the tool were considered clear, valid and suitable for Brazilian culture at all stages of the adaptation process, providing preliminary evidence of content validity.
Both in the pre-test phase and in the evaluation by a panel of expert judges, all the items were evaluated positively, demonstrating that this version of the SETQ Smart is a potentially valid tool for application in Brazil.
References
-
1 Brasil. Decreto nº 7.562, de 15 de setembro de 2011. Dispõe sobre a Comissão Nacional de Residência Médica - CNRM e o exercício das funções de regulação, supervisão e avaliação de instituições que ofertam residência médica e de programas de residência médica. Diário Oficial da União; 16 set 2011. p. 3 [acesso em 21/02/2025]. Disponível em: Disponível em: http://www.abmes.org.br/arquivos/legislacoes/Decreto-7562-2011-09-15.pdf
» http://www.abmes.org.br/arquivos/legislacoes/Decreto-7562-2011-09-15.pdf -
2 Brasil. Resolução CNRM nº 16, de 30 de setembro de 2022. Dispõe sobre estrutura, organização e funcionamento das Comissões de Residência Médica (COREMEs) nas instituições de saúde que oferecem os Programas de Residência Médica (PRMs) e dá outras providências. Brasília: Ministério da Educação, Secretaria de Educação Superior; 2022 [acesso em 21/02/2025]. Disponível em: Disponível em: https://abmes.org.br/arquivos/legislacoes/Resolucao-CNRM-016-2022-09-30.pdf
» https://abmes.org.br/arquivos/legislacoes/Resolucao-CNRM-016-2022-09-30.pdf -
3 Boerebach BCM. Evaluating clinicians’ teaching performance. Perspect Med Educ. 2015;4(5):264-7. https://doi.org/10.1007/s40037-015-0215-7.
» https://doi.org/https://doi.org/10.1007/s40037-015-0215-7 -
4 Ruiz, PFC. Preceptoria em residência médica: uma avaliação sob a perspectiva dos preceptores. Rev Bras Educ Med. 2024;48(4):e116. https://doi.org/10.1590/1981-5271v48.4-2023-0280.
» https://doi.org/https://doi.org/10.1590/1981-5271v48.4-2023-0280 -
5 Carvalho Filho AM, Santos AA, Wyszomirska RMAF, Gauw JH, Gaia IMSRS, Houly RM. Formação na residência médica: visão dos preceptores. Rev Bras Educ Med . 2022;46(2):e052. https://doi.org/10.1590/1981-5271v46.1-20210237.
» https://doi.org/https://doi.org/10.1590/1981-5271v46.1-20210237 -
6 Ferreira IG, Cazella SC, Costa MR. Medical preceptorship: perceptions and perspectives of new family doctors in Brazil. Rev Bras Educ Med . 2023;47(2):1-11. https://doi.org/10.1590/1981-5271v47.2-20220299.ING.
» https://doi.org/https://doi.org/10.1590/1981-5271v47.2-20220299.ING -
7 Ramani S, Leinster S. AMEE Guide nº 34: Teaching in the clinical environment. Med Teach. 2008;30(4):347-64. https://doi.org/10.1080/01421590802061613.
» https://doi.org/https://doi.org/10.1080/01421590802061613 -
8 Johnson CE, Keating JL, Leech M, Congdon P, Kent F, Farlie MK, et al. Development of the Feedback Quality Instrument: a guide for health professional educators in fostering learner-centred discussions. BMC Med Educ. 2021;21(382):1-17. https://doi.org/10.1186/s12909-021-02722-8.
» https://doi.org/https://doi.org/10.1186/s12909-021-02722-8 - 9 Lawall PZM. A formação do preceptor médico em medicina de família e comunidade: uma proposta de diálogo com a andragogia [dissertação]. Brasília: Fundação Oswaldo Cruz; 2019.
-
10 Costa LB, Castro SS, Dourado DXC, Praxedes BS, Mota TC, Tavares TMRL. Adaptação transcultural do questionário EFFECT para português brasileiro. Rev Bras Educ Med . 2021;45(3):1-11. https://doi.org/10.1590/1981-5271v45.3-20200527.
» https://doi.org/https://doi.org/10.1590/1981-5271v45.3-20200527 -
11 Oliveira Filho GR de, Dal Mago AJ, Garcia JHS, Goldschmidt R. An instrument designed for faculty supervision evaluation by anesthesia residents and its psychometric properties. Anesth Analg. 2008;107(4):1316-22. https://doi.org/10.1213/ane.0b013e318182fbdd.
» https://doi.org/https://doi.org/10.1213/ane.0b013e318182fbdd -
12 Miranda PR, Romano VF. Uma proposta de instrumento de avaliação pedagógica da preceptoria para residências em medicina de família e comunidade. Rev Bras Med Fam Comunidade. 2021;16(43):1-9. https://doi.org/10.5712/rbmfc16(43)2680.
» https://doi.org/https://doi.org/10.5712/rbmfc16(43)2680 -
13 Bezerra NC, Lombarts KMJMH, Naiz SCDR, Castanhel FD, Grosseman S. System for Evaluation of Teaching Qualities (SETQ) Smart para o uso no Brasil: versão para residente. Rev Bras Educ Med . 48(3):e076, 2024. doi: https://doi.org/10.1590/1981-5271v48.3-2023-0217.
» https://doi.org/https://doi.org/10.1590/1981-5271v48.3-2023-0217 -
14 O’Sullivan PS, Irby DM. Reframing research on faculty development. Acad Med. 2011;86(4):421-8. https://doi.org/10.1097/ACM.0b013e31820dc058.
» https://doi.org/https://doi.org/10.1097/ACM.0b013e31820dc058 -
15 Heath JK, Dine CJ, Andolsek KM, Burke AE. Teaching the teachers with milestones: using the ACGME milestones model for professional development. J Gr Med Educ. 2021;13:124-6. https://doi.org/10.4300/JGME-D-20-00891.1.
» https://doi.org/https://doi.org/10.4300/JGME-D-20-00891.1 -
16 Johnson CE, Keating JL, Boud DJ, Dalton M, Kiegaldie D, Hay M, et al. Identifying educator behaviours for high quality verbal feedback in health professions education: literature review and expert refinement. BMC Med Educ . 2016;16(96):1-12. https://doi.org/10.1186/s12909-016-0613-5.
» https://doi.org/https://doi.org/10.1186/s12909-016-0613-5 -
17 Cook DA, Brydges R, Ginsburg S, Hatala R. A contemporary approach to validity arguments: a practical guide to Kane’s framework. Med Educ. 2015;49(6):560-75. https://doi.org/10.1111/medu.12678.
» https://doi.org/https://doi.org/10.1111/medu.12678 -
18 Fluit CRMG, Bolhuis S, Grol R, Laan R, Wensing M. Assessing the quality of clinical teachers identification of studies: a systematic review of content and quality of questionnaires for assessing clinical teachers. J Gen Intern Med. 2010;25(12):1337-45. https://doi.org/10.1007/s11606-010-1458-y.
» https://doi.org/https://doi.org/10.1007/s11606-010-1458-y -
19 Tigelaar DE, Dolmans DH, Wolfhagen IH, van der Vleuten CP. The development and validation of a framework for teaching competencies in higher education. High Educ. 2004;48(2):253-68. https://doi.org/10.1023/B:HIGH.0000034318.74275.e4.
» https://doi.org/https://doi.org/10.1023/B:HIGH.0000034318.74275.e4 -
20 Holmboe ES, Ward DS, Reznick RK, Katsufrakis PJ, Leslie KM, Patel VL, et al. Faculty development in assessment: the missing link in competency-based medical education. Acad Med . 2011;86(4):460-7. https://doi.org/10.1097/ACM.0b013e31820cb2a7.
» https://doi.org/https://doi.org/10.1097/ACM.0b013e31820cb2a7 -
21 van der Meulen MW, Smirnova A, Heeneman S, Oude Egbrink MGA, van der Vleuten CPM, Lombarts KMJMH. Exploring validity evidence associated with questionnaire-based tools for assessing the professional performance of physicians: a systematic review. Acad Med . 2019;94(9):1384-97. https://doi.org/10.1097/ACM.0000000000002767.
» https://doi.org/https://doi.org/10.1097/ACM.0000000000002767 -
22 CanMEDS 2000: Extract from the CanMEDS 2000 Project Societal Needs Working Group Report. Med Teach . 2000;22(6):549-54. https://doi.org/10.1080/01421590050175505.
» https://doi.org/https://doi.org/10.1080/01421590050175505 -
23 Lombarts KMJMH, Ferguson A, Hollmann MW, Malling B, Arah OA. Redesign of the System for Evaluation of Teaching Qualities in Anesthesiology Residency Training (SETQ Smart). Anesthesiology. 2016;125(5):1056-65. https://doi.org/10.1097/ALN.0000000000001341.
» https://doi.org/https://doi.org/10.1097/ALN.0000000000001341 -
24 Lombarts KMJMH, Bucx MJL, Arah OA. Development of a system for the evaluation of the teaching qualities of anesthesiology faculty. Anesthesiology . 2009;111(4):709-16. https://doi.org/10.1097/ALN.0b013e3181b76516.
» https://doi.org/https://doi.org/10.1097/ALN.0b013e3181b76516 - 25 Lombarts M, Bucx M, Rupp I, Keijzers P, Kokke S, Schlack W. An instrument for the assessment of the training qualities of clinician-educators. Ned Tijdschr Geneeskd. 2007;151(36):2004-8.
-
26 van der Leeuw R, Lombarts K, Heineman MJ, Arah O. Systematic evaluation of the teaching qualities of obstetrics and gynecology faculty: reliability and validity of the SETQ tools. PLoS One. 2011;6(5):1-7. https://doi.org/10.1371/journal.pone.0019142.
» https://doi.org/https://doi.org/10.1371/journal.pone.0019142 -
27 Beaton DE, Bombardier C, Guillemin F, Ferraz MB. guidelines for the process of cross-cultural adaptation of self-report measures. Spine (Phila Pa 1976). 2000;25(24):3186-91. https://doi.org/10.1097/00007632-200012150-00014.
» https://doi.org/https://doi.org/10.1097/00007632-200012150-00014 - 28 Anunciação L. Conceitos e análises estatísticas com R e JASP. São Paulo: Nila Press; 2021.
-
29 Terwee CB, Prinsen CAC, Chiarotto A, Westerman MJ, Patrick DL, Alonso J, et al. COSMIN methodology for evaluating the content validity of patient-reported outcome measures: a Delphi study. Qual Life Res. 2018;27(5):1159-70. https://doi.org/10.1007/s11136-018-1829-0.
» https://doi.org/https://doi.org/10.1007/s11136-018-1829-0 - 30 R Core Team. A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing; 2020.
- 31 Field A, Miles J, Field Z. Discovering statistics using R. Thousand Oaks: Sage Publications; 2012.
-
32 Alexandre NMC, Coluci MZO. Validade de conteúdo nos processos de construção e adaptação de instrumentos de medidas. Ciência & Saúde Coletiva. 16(7):3061-3068, 2011. https://doi.org/10.1590/S1413-81232011000800006
» https://doi.org/https://doi.org/10.1590/S1413-81232011000800006 - 33 Lynn MR. Determination and quantification of content validity. Nurs Res. 1986;35(6):382-6.
-
34 Debets MPM, Scheepers RA, Boerebach BCM, Arah OA, Lombarts KMJMH. Variability of residents’ ratings of faculty’s teaching performance measured by five- and seven-point response scales. BMC Med Educ . 2020;20(1):325. https://doi.org/10.1186/s12909-020-02244-9.
» https://doi.org/https://doi.org/10.1186/s12909-020-02244-9 -
35 Arah OA, Heineman MJ, Lombarts KMJMH. Factors influencing residents’ evaluations of clinical faculty member teaching qualities and role model status. Med Educ . 2012;46(4):381-9. https://doi.org/10.1111/j.1365-2923.2011.04176.x.
» https://doi.org/https://doi.org/10.1111/j.1365-2923.2011.04176.x -
36 Boerebach BCM, Arah OA, Busch ORC, Lombarts KMJMH. Reliable and valid tools for measuring surgeons’ teaching performance: residents’ vs. self evaluation. J Surg Educ. 2012;69(4):511-20. https://doi.org/10.1016/j.jsurg.2012.04.003.
» https://doi.org/https://doi.org/10.1016/j.jsurg.2012.04.003 -
37 Al Ansari A, Strachan K, Hashim S, Otoom S. Analysis of psychometric properties of the modified SETQ tool in undergraduate medical education. BMC Med Educ . 2017;17(56):1-9. https://doi.org/10.1186/s12909-017-0893-4.
» https://doi.org/https://doi.org/10.1186/s12909-017-0893-4 -
38 Perez-Bret E, Altisent R, Rocafort J. Definition of compassion in healthcare: a systematic literature review. Int J Palliat Nurs. 2016;22(12):599-606. https://doi.org/10.12968/ijpn.2016.22.12.599.
» https://doi.org/https://doi.org/10.12968/ijpn.2016.22.12.599 -
39 Jamtvedt G, Young JM, Kristoffersen DT, Thomson O’Brien MA, Oxman AD. Audit and feedback: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2003;(3). https://doi.org/10.1002/14651858.cd000259.
» https://doi.org/https://doi.org/10.1002/14651858.cd000259 -
40 Paterson C, Paterson N, Jackson W, Work F. What are students’ needs and preferences for academic feedback in higher education? A systematic review. Nurse Educ Today. 2020;85:104236. https://doi.org/https://doi.org/10.1016/j.nedt.2019.104236.
» https://doi.org/https://doi.org/https://doi.org/10.1016/j.nedt.2019.104236 -
41 Loeng S. Self-directed learning: a core concept in adult education. Educ Research Int. 2020;(1):1-12. https://doi.org/https://doi.org/10.1155/2020/3816132.
» https://doi.org/https://doi.org/https://doi.org/10.1155/2020/3816132
Research data is only available upon request.
