Factor analysis of the performance at the medical residency exam, progress test and clerkship rotations performance coefficient

Análise fatorial do desempenho no exame de residência médica, teste de progresso e coeficiente de desempenho nos estágios do internato

Abstract:

Introduction:  The relationships between the students’ performance on medical residency exams and progress tests and medical clerkship rotations are not well established.

Objective:  The objective of this study was to measure the correlations between grades on progress tests and clerkship rotations assessments and the medical residency exam and determine which performance had the strongest correlation with the final medical residency exam.

Methods:  This was a retrospective and longitudinal study with correlation analyses of grades on progress tests from the 1st to 6th year of medical school, the clerkship rotations performance coefficient (5th and 6th years of school) and the final medical residency exam in a cohort of students enrolled in a federal public medical school using factor analysis. Students who performed the progress tests from the 1st to 6th year were included.

Results:  Of 123 students enrolled in the first year of medical school in 2009, 114 (92.7%) performed the progress tests during the six years and were included. The average grades on the progress tests from 1 to 10 were 2.67 (1st year), 3.01 (2nd year), 4.19 (3rd year), 4.01 (4th year), 5.19 (5th year), and 6.38 (6th year). The average grades in the clerkship rotations were 8.32 (5th year) and 8.26 (6th year). The average score on the theoretical medical residency exam was 7.53 and the final result of the medical residency exam was 8.05. Factor analysis detected three domains with greater correlation strength that accounted for 76.3% of the model variance. Component 1 was identified as the coefficient of academic performance (CAP) 5th, CAP 6th and final medical residency exam grades, whereas component 2 was constituted by the grades of the 5th and 6th years progress tests and the third component comprised the progress tests of the 2nd, 3rd and 4th years.

Conclusions:  Grades on the progress tests, the clerkship rotations assessments and the final medical residency exam were correlated. Moreover, the performance during the medical clerkship rotations showed the strongest correlations with medical residency exam grades.

Keywords: Medical Student; Progress Test; Medical Clerkship Rotations; Medical Residency; Factor Analysis

Resumo:

Introdução:  As relações entre o desempenho dos alunos nos exames de residência médica e testes de progresso e os estágios no internato médico não estão bem estabelecidas.

Objetivo:  Este estudo teve como objetivos medir as correlações entre as notas nos testes de progresso e as notas no internato e o resultado final do exame de residência médica, e determinar qual desempenho teve a maior correlação com o exame final da residência médica.

Método:  Trata-se de um estudo retrospectivo e longitudinal com análises de correlação de notas em provas de progresso do primeiro ao sexto ano do curso de Medicina, coeficiente de desempenho de estágios do internato (quinto e sexto anos) e notas do exame final de residência médica em uma coorte de alunos matriculados em uma Faculdade de Medicina de uma instituição pública federal, usando análise fatorial. Foram incluídos os alunos que realizaram os testes de progresso do primeiro ao sexto ano.

Resultado:  Dos 123 alunos matriculados no primeiro ano do curso de Medicina em 2009, 114 (92,7%) realizaram os testes de progresso durante os seis anos letivos e foram incluídos. As notas médias nos testes de progresso de 1 a 10 foram 2,67 (primeiro ano), 3,01 (segundo ano), 4,19 (terceiro ano), 4,01 (quarto ano), 5,19 (quinto ano) e 6,38 (sexto ano). As notas médias nos estágios foram 8,32 (quinto ano) e 8,26 (sexto ano). A nota média no exame teórico da residência médica foi 7,53; e a média no exame final da residência, 8,5. A análise fatorial detectou três domínios com maior força de correlação que responderam por 76,3% da variância do modelo. O componente 1 foi identificado como coeficiente de rendimento acadêmico (CAP) 5º, CAP 6º e o resultado final do exame de residência médica, o componente 2 foi formado pelas notas das provas de progresso do quinto e sextos anos, e o terceiro componente compreendeu as notas do progresso do segundo, terceiro e quarto anos.

Conclusão:  As notas das provas de progresso, as avaliações do internato e o exame final de residência médica apresentaram correlações significantes. Além disso, o desempenho durante o internato apresentou maior correlação com as notas do exame final de residência médica.

Palavras-chave: Estudante de Medicina; Teste de Progresso; Internato Médico; Residência Médica; Análise Fatorial

INTRODUCTION

Progress tests are comprehensive longitudinal assessments used to measure students’ knowledge throughout a curriculum that allow, when properly designed and applied, observation of the individual growth pattern over repeated tests and comparisons of cohorts^1)-(6.

In Brazil, at Escola Paulista de Medicina - Universidade Federal de São Paulo, SP, the progress test was introduced in 1996^7),(8. This assessment has been carried out annually for students from the 1^st to the 6^th year of medical school at several institutions, either alone or in partnership^1),(9. Additionally, numerous studies have been carried out aiming to verify the students’ performance on the progress test^7),(10),(11.

Furthermore, an important question that arises is about the validity of taking the progress test from the 1^st to the 6^th year, taking into consideration the amount of human and financial resources that are spent for its performance, especially for the first years of medical school. In this sense, an attempt to answer this question would be found if the grades obtained during medical school showed correlations between them and any relationship with passing the medical residency exam^12),(13.

Karay and Schauber reported relationships between the individual students’ development assessed by progress tests and performance in high stakes national licensing examinations, showing that the progress tests were suitable to monitor the students’ growth of knowledge during the course of medical training¹⁴.

A study performed at Universidade Federal de São Paulo searched for a predictable statistical model of success in medical residency exams based on the progress test and medical clerkship evaluations. Data on 114 medical students that took the progress tests and had clerkship evaluation at the 5^th and 6^th years were searched for predictors of the student’s score in the medical residency exam by linear regression analysis. The final model showed that the grades of the 5^th year clerkship rotations could predict the medical residency exam score, but grades of the progress test only approached the threshold of significance (p=0.06). Although the progress test failed to predict the final residency score, the univariate analysis pointed out to some correlation between the progress tests performed at the 4^th, 5^th and 6^th years¹⁵.

To investigate these correlations and eventually expand the number of factors influencing the residency exam, the factor analysis, which is a statistical method that searches for correlations between factors and allows the grouping of strongly correlated variables that explain a given phenomenon, would be considered appropriate¹⁶.

Thus, the objective of this study was to measure the correlation between grades on the progress tests, clerkship rotations assessments and medical residency exam and search for the strongest correlation with grades of the final medical residency exam using factor analysis.

METHODS

This was a retrospective and longitudinal study with correlation analyses of grades attained on the progress tests from the 1^st to 6^th year of medical school, the clerkship rotations performance coefficient (5^th and 6^th year) and medical residency exam in a cohort of students enrolled in the first year of medical school at the Universidade Federal de São Paulo in 2009.

The inclusion criteria were students enrolled in the 1^st year in 2009 at this Institution who undertook the progress tests (PT) from the 1^st to the 6^th year. The exclusion criteria were students who missed the progress tests at any time during the six years of the medical school.

Correlations between the grades on the progress tests, clerkship rotations coefficient and medical residency exam of all included students were performed using factor analysis^17),(18.

This study was approved by the institution’s Research Ethics Committee under number 2,555,803.

Progress test

The progress tests at Escola Paulista de Medicina-Universidade Federal de São Paulo were administered once a year in the second semester of the school year. They contained 120 multiple-choice questions with five alternatives and were prepared in conjunction with eight universities participating in the university agreement, covering the entire cognitive content studied until the end of the course, in the areas of basic sciences, internal medicine, surgery, pediatrics, public health, ethics, gynecology and obstetrics. The scores ranged from 0-10, based on the percentage of correct answers. The test was not mandatory, and feedback was given to students by handing over the question booklet and releasing the answer key.

Coefficient of academic performance

During the 5^th year, students’ rotations comprised outpatient internal medicine, family medicine, primary care, interdisciplinary outpatient clinic for adult comprehensive care, cardiology, cardiovascular surgery, endocrinology, gastrointestinal clinic, gastrointestinal surgery, gynecology, obstetrics, hematology, nephrology, urology, neurology, neurosurgery, ophthalmology, otorhinolaryngology, and community pediatrics, pneumology, thoracic surgery and psychiatry.

Throughout the 6^th year, students attend the following areas: general surgical ward, internal medicine ward, infectology, emergency medicine, anesthesiology, plastic surgery, vascular surgery, pediatric emergencies, clinical pediatrics, neonatal pediatrics, obstetrics, orthopedics and traumatology, surgical emergency room, advanced life support in cardiology and an optional clerkship.

The coefficients of academic performance (CAP) for the last two years of medical school that corresponded to the medical clerkship rotations were analyzed.

The CAP is the index that measures, throughout the course, the student’s academic performance at the end of each term and represents the weighted mean of the grades during the clerkship rotations. Each Academic Department or Discipline evaluate their students at the end of the shift, mostly through a written assessment test, skills and attitudes, requiring a frequency of at least 80% of activities and a minimum grade of 6 in 10.

Medical residency exam

This exam was carried out in two stages. The first corresponded to the written theory-based exam with 100 assertive questions with short answers to questions equally distributed in the areas of internal medicine, surgery, obstetrics and gynecology, preventive medicine and pediatrics, and it lasted four hours and thirty minutes.

The second phase comprised the practical test for which students who obtained the highest grades in a ratio of three students/vacancy were invited. The practical test consisted of two stages, one computer-based and the other in the form of practical stations (the Objective Structured Clinical Examination: OSCE). In the computer-based test, the candidates answered 50 questions that involved images displayed on a computer available to each candidate. This hour-long test consisted of questions about internal medicine, general surgery, obstetrics and gynecology, pediatrics and preventive and social medicine. The other stage of the practical test (OSCE) consisted of four stations on clinical medicine, clinical surgery, pediatrics and gynecology-obstetrics. In this test, the candidates performed an activity related to medical practice during five minutes and were observed by teachers who assigned a grade based on a checklist.

The final result of the medical residency exam, which was included in this analysis, was determined by the sum of the scores obtained in the written theoretical test (multiplied by 5), in the practical test (multiplied by 4) and grade given for the interview with the analysis of the candidate’s curriculum and performance during this activity (multiplied by 1).

Statistical analysis

The descriptive analysis of the test scores was expressed as the mean and standard deviation and the median with minimum and maximum values. The average grades on the progress tests from the 1^st to the 6^th year, CAP of the 5^th year and CAP of the 6^th year and the medical residency exam were compared by paired t-test. To analyze data reliability, Cronbach’s alpha¹⁹⁾ was evaluated and considered adequate when greater than 0.7²⁰.

Correlations between scores on all exams included in the study were performed through factor analysis of the grades^17),(18.

The following criteria were established for factor analysis: a sample size of at least 100 grades in each test²¹; minimum communality value of 0.5^16),(18; and Kaiser-Meyer-Olkin (KMO) coefficient greater than 0.6²².

The factor extraction technique was based on the main components¹⁷. The number of factors extracted followed the Kaiser rule, which determined that only factors with an eigenvalue greater than 1 should be extracted to obtain an accumulated explained variance of the extracted factors greater than 60%²¹.

The rotation of the factors was performed using the Varimax orthogonal technique, with the main objective to make the empirical results more easily interpretable, while conserving its statistical properties. This method seeks to minimize the number of variables that show high loads in each factor²².

Finally, the extracted components were identified with their respective factors and each factor loading.

Statistical analyses were performed with the SPSS^® software (IBM SPSS Statistics, Somers, NY, USA, version 17), with a significance level of p < 0.05.

RESULTS

In 2009, 123 students were enrolled in the first year of medical school at the assessed University. Of these, 114 (92.7%) took the progress tests from 2009 to 2014 and had medical clerkship rotations evaluations (CAP 5^th and 6^th) in 2013 and 2014. Of the 114 students included in the study, 105 (92.1%) completed all stages of the residency exams at the University in 2015, and all 105 students passed the exam, thus being included in the analysis (Figure 1).

Figure 1
Flowchart of the study. PT: Progress tests at each year during the six years of Medical School; CAP: Coefficient of academic performance during Medical clerkship rotations; MR FR: Final result of Medical Residency Exam.

Data reliability assessed by Cronbach’s alpha coefficient showed significant internal consistency between the grades on the progress tests from the 1^st to 6^th year, CAP 5^th year, CAP 6^th year and the medical residency exam, with a value of 0.736, 95% CI: 0.653 - 0.806 (p <0.001).

The average test scores are shown in Table 1. There was a progressive increase in the grades on the progress tests from the 1^st to the 6^th year of medical school, with statistically significant differences between all tests (p <0.001), except between the grades on the progress tests in the 3^rd and 4^th years, with a reduction in the grade in the 4^th year (p = 0.028). The average grade for the CAP 5th year was higher than the average for the CAP 6th year (p <0.001), and the grade on the residency exam was lower than the clerkship rotations grades.

Table 1
Descriptive data of grades from the 1^st to 6^th year for the Progress Tests, Coefficient of academic performance during Medical clerkship rotations and the Final Result of the Medical Resident Exam.

	Mean ± SD	Median	Minimum	Maximum
1st PT (n=114)	2.67 ± 0.97	2.83	0	4.5
2nd PT (n=114)	3.00 ± 0.70	3.04	0	4.58
3rd PT (n=114)	4.19 ± 1.13	4.33	0	6.25
4th PT (n=114)	4.01 ± 1.05	4.14	0	6.58
5th PT (n=114)	5.19 ± 1.56	5.42	0	7.25
6th PT (n=114)	6.38 ± 1.80	6.83	0	8.42
CAP 5th (n=114)	8.33 ± 0.29	8.34	7.71	8.96
CAP 6th (n=114)	8.26 ± 0.28	8.29	7.53	8.84
MR T (n=105)	7.53 ± 0.56	7.63	5.75	8.95
MR FR (n=105)	8.05 ± 0.42	8.10	6.84	9.28

• SD: Standard Deviation; PT: Progress Test; CAP: Coefficient of academic performance during Medical clerkship rotations; MR T: Result of the Theoretical Medical Resident Exam; MR FR: Final Result of the Medical Residency Exam.

The analysis of the communalities of the different exams varied from 0.46 to 0.839. The grades on the 1^st year progress test had a coefficient 0.46, suggesting less strength in the linear relationship with the other variables (<0.5) and were, therefore, removed from the factor analysis.

The analysis of the grades on the progress tests in the 2^nd, 3^rd, 4^th, 5^th and 6^th year, grades of the performance at the clerkship rotations (CAP 5^th, CAP 6^th) and grades on the medical residency exam regarding the Kaiser-Meyer-Olkin test (KMO) was 0.709 and the Bartlett’s sphericity test was significant (p <0.001), showing adequacy of the data for the factor analysis.

The extraction of factors, following the Kaiser rule, determined the extraction of three factors with an eigenvalue greater than 1 (Figure 2).

Figure 2
Scree Plot showing the three components with Eigenvalue higher than 1.

The first component consisting of the variables CAP 5^th, CAP 6^th and medical residency grades explained 43.9% of the variance, whereas the second component was represented by the progress tests in the 5^th and 6^th year with 18.0%, and the third component, which consisted of the progress test grades in the 2^nd, 3^rd and 4^th years explained 14.4% of the variance of the model, totaling 76.3% of the total variance (Table 2).

Table 2
Isolated and total variance explained by each matrix component

Initial eigenvalues				Extraction sums of squared loadings
Components	Total	% of Variance	Cumulative %	Total	% of Variance	Cumulative %
1	3.511	43.884	43.884	3.511	43.884	43.884
2	1.443	18.035	61.920	1.443	18.035	61.920
3	1.148	14.350	76.270	1.148	14.350	76.270
4	.634	7.926	84.196
5	.520	6.495	90.692
6	.317	3.963	94.654
7	.271	3.392	98.046
8	.156	1.954	100.000

The matrix rotation using the Varimax method allowed the visualization of the three components with greater correlation strength and generated a new correlation matrix. Component 1 was identified as CAP 5^th, CAP 6^th and medical residency grades, component 2 was formed by the grades on the progress tests in the 5^th and 6^th years and the third component comprised the grades on the progress tests in the 2^nd, 3^rd and 4^th years (Table 3).

Table 3
Correlation matrix for each component after matrix rotation using the Varimax method.

Components	1	2	3
2nd PT	.137	-.092	.795
3rd PT	.065	.465	.709
4th PT	.231	.422	.680
5th PT	.144	.882	.082
6th PT	.102	.884	.150
CAP 5th	.840	.022	.328
CAP 6th	.798	.073	.461
MR FR	.795	.272	-.173

• PT: Progress Test; CAP: Coefficient of academic performance during Medical clerkship rotations; MR FR: Final Result of the Medical Resident Exam. Extraction Method: Principal Component Analysis. Rotation Method: Varimax with Kaiser Normalization. Converged rotation in 5 interactions.

DISCUSSION

The present study showed that, with some variations, the percentage of correct answers on the progress tests from the 1^st to the 6^th year of medical school and the medical residency exam scores at the Universidade Federal de São Paulo were similar to those of other medical schools^3),(9),(11.

Willoughby and Hutcheson²³ reported lower percentages of correct answers than those in the present study on the progress tests from the 1^st to the 6^th year, which were 6.1, 16.1, 30.7, 41.6, 50.9 and 56.0% at the University of Missouri. Likewise, Van der Vleutenet al³⁾ in a 15-year analysis of progress tests at Maastrich Medical School in the Netherlands, from 1977 to 1985, showed increasing percentages of correct answers from the 1^st to the 6^th year, with values ​​close to 20% in the 1^st year and 60% in the 6^th year. Another study also showed similar results on the progress tests for three classes of McMaster University students, with initial scores of 10 and 20% at the beginning of the course, with an almost linear rise until reaching 50% on the 5^th exam².

The theoretical medical residency exam score of 7.5 ± 0.6 was similar to that observed in another Brazilian University, in 2012 (7.6 ± 0.9), 2013 (7.3 ± 0.8), and 2014 (7.5 ± 0.9) in the multiple-choice questions in the areas of general surgery, internal medicine, gynecology and obstetrics, pediatrics and public health¹¹.

The factorial model obtained in the present study showed that although there were correlations between each of the tests, the strongest correlations were detected among three groups of students’ grades. The first group was represented by grades during the clerkship rotations and at the medical residency exam; the second group was represented by grades on the 5^th and 6^th year progress tests; and the third group by the 2^nd, 3^rd and 4^th-year progress tests.

Therefore, this analysis showed that the performance during the clerkship rotations, represented by the coefficients of academic performance during clerkship rotations and 5^th and 6^th year progress tests showed the greatest association with grades on the medical residency exam. The 2^nd, 3^rd and 4^th-year progress tests were also correlated with performance on the medical residency exam, although at a lower level.

Authors have reported an association between the global means of progress tests performed during medical course with the theoretical result of the medical residency exam, but not the final result of the medical residency exam. This fact could be explained because the progress test assesses the formative content, as well as the theoretical medical residency exam¹³. However, the present study aimed to search for factors correlated to the final result of medical residency exam, which measures a summative dimension. For this reason, the progress test grades, as well as the evaluation of the clerkship rotation had to be included in the factorial analysis, as clerkship evaluation represents a global knowledge^2),(24. According to the factor analysis, the final result of the medical residency exam was correlated with both the progress tests from the 2^nd to 6^th year and clerkship. The difference found between this study and the literature might be explained by the variables included in the analysis. As the final result of the medical residency exam included a summative evaluation of knowledge, it could explain the strong correlation with clerkship evaluation and the weaker correlation with the progress test².

After applying the factorial analysis technique, the 1^st year progress test was excluded from the analysis, possibly because of lower grades in the first year than in the subsequent years of the course, which resulted in a low correlation between them. Likewise, a study showed a significant Pearson’s correlation between the final result of the medical residency exam and the progress test scores, except for the 1^st, 2^nd and 3^rd years¹⁵. Other authors have already reported lower percentages of correct answers in the initial years with an almost linear rise towards the end of the course^2),(3),(23.

It should be noted that the reason why the 1^st year progress test was not included in the model was that its correlation was not as strong as the other tests and the fact that its communality was slightly lower than 0.5. Such technical issues of factor analysis are not absolutely rigid; however, it served to obtain a better model from the mathematical point of view. Thus, a more liberal approach in relation to the inclusion of the 1^st year progress test grades could have shown a correlation with less power in the explained variance and, perhaps, created a fourth component, with much less strength than the others. This analysis is in line with the idea that the progress test among first-year students has less discriminatory power because only a small percentage of the questions refer to the knowledge acquired at the beginning of the course³ and, furthermore, in the 1^st year of medical school the adhesion to progress might be lower than in more advanced years, with lower scores, which possibly reduced the strength of the correlation with later assessments²⁵.

Therefore, these results do not imply that grades of the 1^st year progress test do not provide any added value to the students’ performance on the medical residency exam. In this line, a study reported that initial levels and gains of knowledge during the medical course are related to later performances on national licensing examinations. The authors showed that the pre-clinical (1^st-4^th semesters) and the clinical phase (5^th-12^th semesters) of the medical course were both positively related to long-term assessments¹⁴.

Therefore, it can be said that the progress test would be important to understand the performance evaluation process of medical students at each year, to analyze the development of students within a particular cohort, so it could be used as a tool for early diagnosis and interventions, as well as for comparisons between medical schools^{1),(14),(24),(26}.

Therefore, considering all the research together, although in the present study, grades of the 1^st year progress test were not correlated with performance in other tests, students should be encouraged to take these tests in a committed way in the very beginning of the course. Moreover, efforts to improve the quality of the progress test and measures to improve students’ adhesion to it, especially in the first years, and better quality of the progress test would increase the correlation between these assessments^{2),(24),(25),(27}.

A limitation of this study was the analysis of only one group of students who started medical school in a given year. Additionally, except for the progress test, which could be interinstitutional, the inclusion of only one institution that may show clerkship rotations heterogeneity, different contexts of the practice of medicine and different forms of assessments compared to other schools of medicine, could represent an inclusion bias^28),(29, supporting the need for further studies.

The strength of this research was the longitudinal analysis of the test scores performed during the course, with great adherence by students to the progress tests and adequate reliability of data consistent with the internal consistency coefficient found in this study. Furthermore, the statistical treatment that was developed here allowed measuring the strength of the correlation factor, which might be in accordance with the daily experience. In addition, unlike other studies, the present research showed that the progress test applied in the last five years of medical school at our institution was also correlated with the final result of the medical residency exam, even though more studies are needed to validate these results.

CONCLUSIONS

This study showed that grades of the progress test from the 2^nd to 6^th year, clerkship rotations assessments and the final medical residency exam had strict and positive correlations between them. Furthermore, the academic performance during medical clerkship rotations showed the strongest correlations with the medical residency exam scores.

REFERENCES

• ¹ Reberti AG, Monfedini NH, Ferreira Filho OF, Andrade DF, Pinheiro CEA, Silva JC. Progress test in medical school: a systematic review of the literature. Rev Bras Educ Med 2020;44(1):e014.
• ² Blake JM, Norman GR, Keane DR, Mueller CB, Cunnington J, Didyk N. Introducing progress testing in MacMaster University’s problembased medical curriculum: psychometric properties and effects on learning. Acad Med. 1996;71(9):1002-7.
• ³ Van der Vleuten CP, Verwijnen GM, Wijnen W. Fifteen years of experience with progress testing in a problem-based learning curriculum. Med Teach. 1996;18(2):103-9.
• ⁴ Langer MM, Swanson DB. Practical considerations in equating progress tests. Med Teach. 2010;32(6):509-12.
• ⁵ Al Alwan I, Al-Moamary M, Al-Attas N, Al Kushi A, AlBanyan E, Zamakhshary M, et al. The progress test as a diagnostic tool for a new PBL curriculum. Educ Health. 2011;24(3):493-503.
• ⁶ Findyartini A, Werdhani RA, Iryani D, Rini EA, Kusumawati R, Poncorini E, et al. Collaborative progress test (cPT) in three medical schools in Indonesia: the validity, reliability and its use as a curriculum evaluation tool. Med Teach . 2015;37(4):366-73.
• ⁷ Faccin MP. O teste do progresso como instrumento de avaliação da aquisição do conhecimento na graduação médica [tese]. São Paulo: Universidade Federal de São Paulo; 2004.
• ⁸ Borges DR, Stella RC. Avaliação do ensino de Medicina na Universidade Federal de São Paulo. Rev Bras Educ Med . 1999;23;(1):11-7.
• ⁹ Bicudo AM, Hamamoto Filho PT, Abbade JF, Hafner ML, Maffei CM. Consortia of cross-institutional progress testing for all medical schools in Brazil. Rev Bras Educ Med . 2019;43(4):151-6.
• ¹⁰ Sakai MH, Ferreira Filho OF, Almeida MJ, Mashima DA, Marchese MC. Teste de progresso e avaliação do curso: dez anos de experiência da medicina da Universidade Estadual de Londrina. Rev Bras Educ Med . 2008;32(2):254-63.
• ¹¹ Ferreira RC. Relação entre o desempenho no teste de progresso e na seleção para residência médica [tese]. Campinas: Universidade Estadual de Campinas; 2019.
• ¹² Haghdoost AA, Esmaeili A. The validity of medical students’ scores in their internship courses: a historical cohort study. J Med Edu. 2008;12(1 & 2):e105368.
• ¹³ Hamamoto Filho PT, de Arruda PL, do Valle AP, Abbade JF, Bicudo AM. The correlation between students’ progress testing scores and their performance in a residency selection process. Med Sci Ed. 2019;29(8):1071-5.
• ¹⁴ Karay Y, Schauber SK. A validity argument for progress testing: examining the relation between growth trajectories obtained by progress tests and national licensing examinations using a latent growth curve approach. Med Teach . 2018;40(11):1123-9.
• ¹⁵ Andrade MC, Strufaldi MWL, Ferreira RG, Prado GFD, Puccini RF, Santos AMND. Factors associated with student performance on the medical residency test. Rev Assoc Med Bras. 2020;66(10):1376-82.
• ¹⁶ Carvalho FRD. Análise fatorial [tese]. Coimbra: Universidade de Coimbra; 2013.
• ¹⁷ Treiblmaier H, Filzmoser P. Exploratory factor analysis revisited: how robust methods support the detection of hidden multivariate data structures in IS research. Inf Manage. 2010;47(4):197-207.
• ¹⁸ Figueiredo Filho B, Silva Júnior JA. Visão além do alcance: uma introdução à análise fatorial. Opin Pública. 2010;16(1):160-85.
• ¹⁹ Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika. 1951;16:297-34.
• ²⁰ Streiner DL. Being inconsistent about consistency: when coefficient alpha does and doesn’t matter. J Pers Assess. 2003;80(3):217-22.
• ²¹ Hair Jr, Black WC, Babin BJ, Anderson RE, Tatham RL. Multivariate data analysis. 6th ed. Upper Saddle River, NJ: Pearson Prentice Hall; 2006.
• ²² Pallant J. SPSS Survival Manual. A step-by-step guide to data analysis using SPSS for Windows. 3rd ed. Maidenhead: Open University Press; 2007.
• ²³ Willoughby TL, Hutcheson SJ. Edumetric validity of the quarterly profile examination. Educ Psychol Meas. 1978;38(4):1057-61.
• ²⁴ Albanese M, Case SM. Progress testing: critical analysis and suggested practices. Adv Health Sci Educ Theory Pract. 2016;21:221-34.
• ²⁵ Sartor LB, da Rosa LL, Madeira K, Uggioni ML, Ferreira Filho OF, da Rosa M I. Percepção dos acadêmicos de Medicina sobre o teste de progresso. Rev Bras Educ Med . 2020;44(2):e062.
• ²⁶ Da Rosa MI, Isoppo CC, Cattaneo HD, Madeira K, Adami F, Ferreira Filho OF. Progress testing as an indicator for improvements in a medical school. Rev Bras Educ Med . 2017;41(1):58-68.
• ²⁷ Hamamoto Filho PT, Bicudo AM. Improvement of faculty’s skills on the creation of items for progress testing through feedback to item writers: a successful experience. Rev Bras Educ Med . 2020;44(1):e018.
• ²⁸ Domingues RC, Amaral E, Zeferino AMB, Antonio MA, Nadruz W. Competência clínica de alunos de Medicina em estágio clínico: comparação entre métodos de avaliação. Rev Bras Educ Med . 2010;34(1):124-31.
• ²⁹ Mota LR, Nunes HR, Martin LC, Hokama PO, Hamamoto Filho PT, Hokama N. “Nota de conceito global” na avaliação da performance do interno de medicina: uma oportunidade desperdiçada. Rev Bras Educ Med . 2020;44(2):e051.

• 2
  Evaluated by double blind review process.
• SOURCES OF FUNDING
  The authors declare no sources of funding for this study.

Edited by

• Chief Editor: Rosiane Viana Zuza Diniz. Associate editor: Pedro Tadao Hamamoto Filho.

Publication Dates

• Publication in this collection
  07 Nov 2022
• Date of issue
  2022

History

• Received
  10 Mar 2022
• Accepted
  17 July 2022