An interdisciplinary team composed of more than thirty people has been engaged in the process of conceiving, designing and validating an online placement test with formative orientation, called SELF (Système d’Evaluation en Langues à visée Formative). SELF is a large-scale assessment device, anchored on the CEFR descriptors and validated according to the ALTE cycle, with both quantitative (CTT – Classical Test Theory and IRT – Item Response Theory) and qualitative methods (questionnaires, interview and focus group), developed within the frame of the ANR IDEFI project Innovalangues. Nowadays, SELF has already placed around 60,000 students in six different languages (Italian, English, French, Mandarin, Japanese and Spanish). Designing a multilingual test with these features is a very challenging and long process. The most challenging aspects to manage concerned i) the respect of the same communicative construct for the six different languages of the test; ii) the necessity to improve item writers’ skills in psychometrics and, more broadly, to spread a good-quality culture of evaluation; iii) the infrastructural and technical side; iv) the necessity to coordinate a large and heterogeneous team for a long time (6 years). Facing these difficulties required the adoption of specific strategies, in order to reach the goal, such as: a very careful organization of the working team, composed of scientific manager, team coordinators and item-writers; the decision to start with two piloting languages, Italian and English, followed by the other four; the redaction and sharing of common documents to guarantee the interlinguistic transfer; the in-house conception of a multi-task platform serving as authoring tool, piloting and pretesting repository, large-scale administration device for tracking, saving and communicating final results.

Designing a Multilingual Large-Scale Placement Test with a Formative Perspective: A Case Study at the University of Grenoble Alpes

Cristiana Cervini
Primo
;
2021

Abstract

An interdisciplinary team composed of more than thirty people has been engaged in the process of conceiving, designing and validating an online placement test with formative orientation, called SELF (Système d’Evaluation en Langues à visée Formative). SELF is a large-scale assessment device, anchored on the CEFR descriptors and validated according to the ALTE cycle, with both quantitative (CTT – Classical Test Theory and IRT – Item Response Theory) and qualitative methods (questionnaires, interview and focus group), developed within the frame of the ANR IDEFI project Innovalangues. Nowadays, SELF has already placed around 60,000 students in six different languages (Italian, English, French, Mandarin, Japanese and Spanish). Designing a multilingual test with these features is a very challenging and long process. The most challenging aspects to manage concerned i) the respect of the same communicative construct for the six different languages of the test; ii) the necessity to improve item writers’ skills in psychometrics and, more broadly, to spread a good-quality culture of evaluation; iii) the infrastructural and technical side; iv) the necessity to coordinate a large and heterogeneous team for a long time (6 years). Facing these difficulties required the adoption of specific strategies, in order to reach the goal, such as: a very careful organization of the working team, composed of scientific manager, team coordinators and item-writers; the decision to start with two piloting languages, Italian and English, followed by the other four; the redaction and sharing of common documents to guarantee the interlinguistic transfer; the in-house conception of a multi-task platform serving as authoring tool, piloting and pretesting repository, large-scale administration device for tracking, saving and communicating final results.
2021
Challenges in Language Testing Around the World
243
253
Cristiana Cervini; Monica Masperi
File in questo prodotto:
File Dimensione Formato  
Challenges 18 Cervini and Masperi-revised-270820-CLEAN-copertina.pdf

Open Access dal 19/02/2023

Tipo: Postprint
Licenza: Licenza per accesso libero gratuito
Dimensione 483.24 kB
Formato Adobe PDF
483.24 kB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11585/656600
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 0
  • ???jsp.display-item.citation.isi??? ND
social impact