Statistical evaluation of research performance of young university scholars: A case study

Avaliação estatística do desempenho em pesquisa de jovens estudantes universitários: estudo de caso

Abstract

The research performance of a small group of 49 young scholars, such as doctoral students, postdoctoral and junior researchers, working in different technical and scientific fields, was evaluated based on 11 types of research outputs. The scholars worked at a technical university in the fields of Civil Engineering, Ecology, Economics, Informatics, Materials Engineering, Mechanical Engineering, and Safety Engineering. Principal Component Analysis was used to statistically analyze the research outputs and its results were compared with factor and cluster analysis. The metrics of research productivity describing the types of research outputs included the number of papers, books and chapters published in books, the number of patents, utility models and function samples, and the number of research projects conducted. The metrics of citation impact included the number of citations and h-index. From these metrics – the variables – the principal component analysis extracted 4 main principal components. The 1^st principal component characterized the cited publications in high-impact journals indexed by the Web of Science. The 2^nd principal component represented the outputs of applied research and the 3^rd and 4^th principal components represented other kinds of publications. The results of the principal component analysis were compared with the hierarchical clustering using Ward’s method. The scatter plots of the principal component analysis and the Mahalanobis distances were calculated from the 4 main principal component scores, which allowed us to statistically evaluate the research performance of individual scholars. Using variance analysis, no influence of the field of research on the overall research performance was found. Unlike the statistical analysis of individual research metrics, the approach based on the principal component analysis can provide a complex view of the research systems.

Keywords
Metrics research; Multivariate analysis; Research performance evaluation; Young scholars