An interdisciplinary team composed of more than thirty people has been engaged in the process of conceiving, designing and validating an online placement test with formative orientation, called SELF (Système d’Evaluation en Langues à visée Formative). SELF is a large-scale assessment device, anchored on the CEFR descriptors and validated according to the ALTE cycle, with both quantitative (CTT – Classical Test Theory and IRT – Item Response Theory) and qualitative methods (questionnaires, interview and focus group), developed within the frame of the ANR IDEFI project Innovalangues. Nowadays, SELF has already placed around 60,000 students in six different languages (Italian, English, French, Mandarin, Japanese and Spanish). Designing a multilingual test with these features is a very challenging and long process. The most challenging aspects to manage concerned i) the respect of the same communicative construct for the six different languages of the test; ii) the necessity to improve item writers’ skills in psychometrics and, more broadly, to spread a good-quality culture of evaluation; iii) the infrastructural and technical side; iv) the necessity to coordinate a large and heterogeneous team for a long time (6 years). Facing these difficulties required the adoption of specific strategies, in order to reach the goal, such as: a very careful organization of the working team, composed of scientific manager, team coordinators and item-writers; the decision to start with two piloting languages, Italian and English, followed by the other four; the redaction and sharing of common documents to guarantee the interlinguistic transfer; the in-house conception of a multi-task platform serving as authoring tool, piloting and pretesting repository, large-scale administration device for tracking, saving and communicating final results.
Cristiana Cervini, Monica Masperi (2021). Designing a Multilingual Large-Scale Placement Test with a Formative Perspective: A Case Study at the University of Grenoble Alpes. New York : Springer [10.1007/978-981-33-4232-3_18].
Designing a Multilingual Large-Scale Placement Test with a Formative Perspective: A Case Study at the University of Grenoble Alpes
Cristiana CerviniPrimo
;
2021
Abstract
An interdisciplinary team composed of more than thirty people has been engaged in the process of conceiving, designing and validating an online placement test with formative orientation, called SELF (Système d’Evaluation en Langues à visée Formative). SELF is a large-scale assessment device, anchored on the CEFR descriptors and validated according to the ALTE cycle, with both quantitative (CTT – Classical Test Theory and IRT – Item Response Theory) and qualitative methods (questionnaires, interview and focus group), developed within the frame of the ANR IDEFI project Innovalangues. Nowadays, SELF has already placed around 60,000 students in six different languages (Italian, English, French, Mandarin, Japanese and Spanish). Designing a multilingual test with these features is a very challenging and long process. The most challenging aspects to manage concerned i) the respect of the same communicative construct for the six different languages of the test; ii) the necessity to improve item writers’ skills in psychometrics and, more broadly, to spread a good-quality culture of evaluation; iii) the infrastructural and technical side; iv) the necessity to coordinate a large and heterogeneous team for a long time (6 years). Facing these difficulties required the adoption of specific strategies, in order to reach the goal, such as: a very careful organization of the working team, composed of scientific manager, team coordinators and item-writers; the decision to start with two piloting languages, Italian and English, followed by the other four; the redaction and sharing of common documents to guarantee the interlinguistic transfer; the in-house conception of a multi-task platform serving as authoring tool, piloting and pretesting repository, large-scale administration device for tracking, saving and communicating final results.File | Dimensione | Formato | |
---|---|---|---|
Challenges 18 Cervini and Masperi-revised-270820-CLEAN-copertina.pdf
Open Access dal 19/02/2023
Tipo:
Postprint
Licenza:
Licenza per accesso libero gratuito
Dimensione
483.24 kB
Formato
Adobe PDF
|
483.24 kB | Adobe PDF | Visualizza/Apri |
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.