Ranking Departments based on research quality: a statistical evaluation of the ISPD indicator

Given the relevance of performance-based funding for the Italian University System, in this paper, we analyse the statistical properties of the ISPD indicator used to provide a ranking of the best 180 Italian Departments. Besides the adoption of a standardization procedure and subsequent aggregation, the main flow of the ISPD indicator appears to be its tendency to polarization, which has important consequences on the ability of such indicator to effectively discriminate between outstanding Departments as opposed to those just above the average. Implications on funding allocation are discussed using data on the 2011-2014 Italian research assessment exercise and some proposals on alternative indicators that solve some of the critical aspects of ISPD are sketched.