Modelos de valor agregado no ensino superior: uma contextualização histórica da experiência brasileira

Fernandes, Vivian Duarte Couto; Miranda, Gilberto José; Alexander, Nicola

doi:10.24109/2176-6681.rbep.101i259.4469

Acessibilidade / Reportar erro

Brasil

Revista Brasileira de Estudos Pedagógicos

Español English

Brasil

Español English

sumário « anterior atual seguinte »

Sumário

ESTUDOS • Rev. Bras. Estud. Pedagog. 101 (259) • Sep-Dec 2020 • https://doi.org/10.24109/2176-6681.rbep.101i259.4469 copiar

Modelos de valor agregado no ensino superior: uma contextualização histórica da experiência brasileira

Autoria SCIMAGO INSTITUTIONS RANKINGS

Resumo:

A avaliação da qualidade das escolas tem sido objeto de pesquisas de economistas, educadores, formuladores de políticas e vários stakeholders em todo o mundo. No Brasil, o Sistema Nacional de Avaliação do Ensino Superior (Sinaes) é uma iniciativa que busca avaliar os programas de graduação do País, seus professores e o desempenho dos estudantes, além de fornecer indicadores de qualidade que explicam a diferença entre esses programas. Um deles é o Indicador de Diferença entre Desempenhos Observado e Esperado (IDD), que mede a contribuição de um programa de graduação para o desempenho dos seus estudantes. Desde 2006, os legisladores vêm mudando a metodologia de cálculo do referido indicador, buscando aprimorá-la como uma medida precisa do valor agregado. Esta pesquisa tem como objetivo discutir essas alterações e seus impactos no ranking dos cursos de graduação em Contabilidade no Brasil. Um design quantitativo foi utilizado para testar quatro modelos de valor agregado registrados nos históricos do IDD e para identificar o impacto dessas mudanças metodológicas. A amostra foi composta por 30.668 estudantes de 911 cursos de Ciências Contábeis. Os resultados mostram que o modelo atual é mais preciso do que os anteriores. No entanto, tanto a literatura quanto os resultados indicam que o modelo pode ser aprimorado com a introdução de variáveis explicativas para o desempenho acadêmico que não podem ser controladas por instituições de ensino superior. O IDD ainda é inadequado porque considera todas as instituições e todos os estudantes como iguais.

Palavras-chave:
eficácia escolar; ensino superior; qualidade do ensino superior

Variable	Until 2010	2011-2012	2013 - date
Doc (Faculty members with Ph.D.s)	0.20	0.15	0.15
Me (Faculty members with master’s degrees)	0.05	0.075	0.075
RT (Full-time professors)	0.05	0.075	0.075
IFF (infrastructure)	0.05	0.075	0.05
ODP (pedagogic teaching structure)	0.05	0.075	0.075
OAF (opportunity for further training/learning)	-	-	0.025
IDD (value-added)	0.30	0.35	0.35
C (Average of seniors' Enade score)	0.15	0.20	0.20
I (Average of freshmen Enade score)	0.15	-	-

Level of analysis	Name of variable (code)	Type of variable	Value of variable
Response Variable	Enade score (C)	Numerical	0 to 100
Explanatory variable (student-related)	Mean Enem score ()	Numerical	0 to 1000
	Score in Enem’s ‘Natural Sciences’ section (CN)	Numerical	0 to 1000
	Score in Enem’s ‘Humanities’section (CH)	Numerical	0 to 1000
	Score in Enem’s ‘Mathematics’section (MT)	Numerical	0 to 1000
	Score in Enem’s ‘Languages’ section (LT)	Numerical	0 to 1000
	Ratio of parents with higher education (w)	Numerical	0 to 1
Explanatory variable (program-related)	Faculty members with Ph.Ds (Doc)	Numerical	1 to 5
	Faculty members with master’s degrees (Me)	Numerical	1 to 5
	Faculty members under full-time employment contracts (RT)	Numerical	1 to 5
	Program’s infrastructure (IFF)	Numerical	1 to 5
	Program’s pedagogical teaching structure (ODP)	Numerical	1 to 5
	Program’s opportunity for further training/learning (OAF)	Numerical	1 to 5

Program ranking	Value-added estimate compared to the sample mean	Index	Interpretation
Level 5: Most effective	At least 2 standard errors above	Higher than or equal to 2.00	Significant evidence that students exceeded the Growth Standard.
Level 4:Above-average effectiveness	Between 1 and 2 standard errors above	Between 0.99 and 1.99	Moderate evidence that students exceeded the Growth Standard.
Level 3: Average effectiveness	Between 1 standard error above and 1 standard error below	Between -1.00 and 1.00	Evidence that students met the Growth Standard.
Level 2: Approaching average effectiveness	Between 1 and 2 standard errors below	Between -1.99 and -0.99	Moderate evidence that students did not meet the Growth Standard.
Level 1: Least effective	More than 2 standard errors below	Less than or equal to -2.00	Significant evidence that students did not meet the Growth Standard.

Variable	Model 1	Model 2	Model 3	Model 4
Intercept	-32.46*	-31.92*	-32.06*	-8.09*
CN (Score in Enem’s ‘Natural Sciences’sections)	0.02*	-	-	0.02*
CH (Score in Enem’s ‘Humanities’ sections)	0.02*	-	-	0.02*
LT (Score in Enem’s ‘Languages’ sections)	0.04*	-	-	0.03*
MT (Score in Enem’s ‘Mathematics’ sections)	0.05*	-	-	0.02*
Enem score	-	0.13*	0.13*	-
W (ratio of students whose parents have higher education)	-2.76**	-2.45	-2.40	-
Doc ( faculty members with Ph.D.s)	0.02	0.06	0.06	-
Me (faculty members with master’s degrees)	0.18	0.18	0.18	-
RT (full-time professors)	0.39*	0.32*	0.32*	-
IFF (infrastructure)	0.78*	1.07*	1.17*	-
ODP (pedagogical teaching structure)	0.53**	0.31	0.37	-
OAF (opportunity for further training/learning)	-	-	-0.17	-

	Student level			Institution level
	Enade	General training	Specific training	Enade	General training	Specific training
Enem	0.54	0.37	0.51	0.71	0.61	0.68
CN	0.44	0.31	0.41	0.63	0.56	0.60
CH	0.43	0.31	0.40	0.60	0.55	0.56
LT	0.45	0.33	0.41	0.66	0.57	0.62
MT	0.44	0.26	0.43	0.69	0.53	0.67

	Model 1	Model 2	Model 3
Model 1	1.00	-	-
Model 2	0.98	1.00	-
Model 3	0.98	1.00	1.00
Model 4	0.76	0.78	0.78

Groups	Undergraduate programs	Sum	Mean	Variance
IDD_M1	911	2023.79	2.22a	0.48
IDD_M2	911	1952.97	2.14a	0.52
IDD_M3	911	1943.91	2.13a	0.51
IDD_M4	911	2152.04	2.36b	0.21

CPC (continuous)	CPC (categorical)
0 ≤ CPC < 0.945	1
0.945 ≤ CPC < 1.945	2
1.945 ≤ CPC < 2.945	3
2.945 ≤ CPC < 3.945	4
3.945 ≤ CPC ≤ 5	5

CPC levels	Model 1	Model 2	Model 3	Model 4
Level 1	4	5	5	5
Level 2	145	160	161	101
Level 3	593	586	586	639
Level 4	165	156	155	161
Level 5	4	4	4	5

Effectiveness level	Model 1	Model 2	Model 3	Model 4
Level 1 - Least effective	14	15	16	78
Level 2 - Above average effectiveness	68	59	57	110
Level 3 - Average effectiveness	578	587	587	532
Level 4 - Above average effectiveness	192	189	189	102
Level 5 - Most effective	59	61	62	89

	2-level down-grade	2-levels upgrade	t test (p-value)	Interpretation (The most effective programs have…)
Enade mean	42.71	50.96	0.00	Highest overall score on the Enade
Mean Enade score for FG	52.91	58.53	0.00	Highest Enade score for FG
Mean Enade score for CE	39.28	48.41	0.00	Highest Enade score for CE
Mean books read/year	0.43	0.55	0.05	Highest % of people eading 3 books or +
Mean hours of study	0.28	0.51	0.00	Highest % of people studying above 3h
Mean Enem score for CN	490.83	501.31	0.05	Highest Enem score for Natural Science
Mean Enem score for CN	509.80	540.18	0.00	Highest Enem score for History
Mean Enem score for LT	533.45	546.44	0.00	Highest Enem score for Literature
Mean Enem score for LT	556.93	574.22	0.00	Highest Enem score for Math
Mean IGC (continuous)	2.52	3.00	0.00	Highest general Index
Mean ODP	2.26	4.29	0.00	Highest pedagogical teaching
Mean IFF	2.27	4.48	0.00	Highest infrastructure
Mean OAF	2.26	4.20	0.00	Highest opportunity for further training/learning
Mean Me	2.87	3.99	0.00	Highest % of faculty members with masters degrees
Mean Doc	1.16	1.84	0.04	Highest % of faculty members with Ph.D.s

		IDD_M1	IDD_M2	IDD_M3	IDD_M4
Marital status	ρpb	0.20*	0.20*	0.20*	0.04*
Ethnicity	ρpb	0.05*	0.06*	0.06*	0.04*
Nationality	ρpb	0.02*	0.02*	0.02*	0.01*
Level of education (father’s)	ρpb	-0.02*	-0.02*	-0.01*	0.00
Level of education (mother’s)	ρpb	-0.01*	-0.01*	-0.01	0.00
Income	ρpb	0.06*	0.06*	0.06*	0.03*
Work	ρpb	-0.00	0.00	0.00	0.02*
Funding	ρpb	-0.17*	-0.19*	-0.19*	0.01
Affirmative policy	ρpb	-0.04*	-0.04*	-0.04*	-0.00
High school institution	ρpb	0.01	0.01*	0.01*	0.01
Type of high school	ρpb	0.12*	0.12*	0.12*	0.02*
First in the family in a HEI	ρpb	-0.02*	-0.02*	-0.02*	-0.01
No. of books read	ρpb	-0.02*	-0.02*	-0.02*	-0.02*
Hours of study	ρpb	-0.05*	-0.04*	-0.05*	-0.05*
Enem (CN - Natural Sciences)	ρ	0.03*	0.02*	0.02*	0.04*
Enem (CH - Humanities)	ρ	0.03*	0.01	0.01	0.03*
Enem (LT - Languages)	ρ	0.04*	0.04*	0.04*	0.04*
Enem (MT - Mathematics)	ρ	0.03*	0.05*	0.04*	0.06*

		IDD_M1	IDD_M2	IDD_M3	IDD_M4
Academic organization	ρpb	0.02	0.02	0.02	0.11*
No. of programs in the HEI	ρ	0.14*	0.16*	0.16*	0.24*
IGC (continuous)	ρ	0.35*	0.34*	0.34*	0.42*
Learning modality	ρpb	0.11*	0.04	0.04	0.05
ODP	ρ	0.35*	0.35*	0.35*	0.26*
IFF	ρ	0.36*	0.38*	0.38*	0.31*
OAF	ρ	0.35*	0.36*	0.36*	0.30*
Faculty members with master’s degrees	ρ	0.12*	0.13*	0.13*	0.18*
Faculty members with Ph.D.s	ρ	0.07*	0.08*	0.07*	0.11*
Employment contract	ρ	0.15*	0.13*	0.13*	0.16*

Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira Setor de Indústrias Gráficas - Quadra 04 - lote 327, Térreo, Ala B, CEP 70.610-440 Brasília-DF Brasil, Telefones: (61) 2022-3077, 2022-3078 - Brasília - DF - Brazil
E-mail: editoria.rbep@inep.gov.br

Acompanhe os números deste periódico no seu leitor de RSS

[TFN1] Source: Elaborated by the authors based on Brasil. Inep (2009aBRASIL. Lei nº 10.861, de 14 de abril de 2004. Institui o Sistema Nacional de Avaliação da Educação Superior - Sinaes e dá outras providências. Diário Oficial da União, Brasília, DF, 15 abr. 2004. Seção 1, p. 3.), Brasil. Inep. (2012BRASIL. Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira (Inep). Diretoria de Avaliação da Educação Superior (DAES). Nota Técnica nº 29, de 15 de outubro de 2012. Cálculo do Conceito Preliminar de Curso (CPC) referente ao ano de 2011. Brasília, DF: Inep , 2012. Disponível em: <Disponível em: http://download.inep.gov.br/educacao_superior/enade/notas_tecnicas/2011/nota_tecnica_indicadores_2011_2.pdf >. Acesso em: 20 nov. 2020.
http://download.inep.gov.br/educacao_sup... ); Brasil. Inep. (2014aBRASIL. Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira (Inep). Nota Técnica nº 72. Cálculo do conceito preliminar de curso referente a 2013. Brasília, DF: Inep , 2014a. Disponível em: <Disponível em: http://download.inep.gov.br/educacao_superior/enade/notas_tecnicas/2013/nota_tecnica_n_72_2014_calculo_cpc_2013.pdf >. Acesso em: 27 nov. 2020.
http://download.inep.gov.br/educacao_sup... ).