Methodological Challenges in Assessment of Internal and External Effectiveness of Italian Higher Education Through the AlmaLaurea Surveys

AlmaLaurea is a consortium of 64 Italian universities that together account almost 80% of all Italian bachelor’s and master’s degree-earners. AlmaLaurea is a fully integrated information system in which a graduate data-base represents the core of a large integrated set of research activities and services which provide an essential reference point for anyone who deals with issues involving higher education, graduate employment and youth conditions. AlmaLaurea’s chief research efforts comprise tracking studies providing information on both university graduate “profiles” (family background. upper secondary education, type and duration of academic studies, study conditions, academic performance, work experience, satisfaction with the university experience, foreign language and IT skill acquisition, internships, further study opportunities, job perspectives) and graduates’ employment conditions at various points in time after graduation (postgraduate training, employment conditions, access to the labor market, current job characteristics, employer characteristics, use and need for a degree in the current job, degree effectiveness, job search strategies). This set of two initiatives comprises a major panel study, in that for all member universities the aim is to contact and collect information on every single graduate at the time of graduation, one year after earning his/her degree, three years after graduation and again five years after graduation, for a total of four points in time. The annual survey findings perform many functions, mainly having to do with providing national and local governments and member universities with reliable and up-to-date information on the evolution of graduates’ post-academic careers and labor market performances in order to carry out assessments of specific degree programs and higher education institutions. The findings also represent a tool for helping aspiring university students and their families make informed choices, as well as employers gauge degree program effectiveness. The most recent “profile” survey, involving individuals’ graduating during the solar year 2010, involves approximately 191 thousand degree-earners. The most recent “employment conditions” survey, carried out in 2011, involved almost 410 thousand graduates. Over the last decade the execution of these studies has been accompanied by a set of major methodological challenges. Since 2005 AlmaLaurea has abandoned its CATI (computer-assisted telephone interviewing)-only approach to data collection and adopted an integrated CATI+CAWI (computer-assisted web interviewing) strategy, in part in response to these challenges, in part posing new questions, which are addressed in the text