We introduce the second edition of the CheckThat! Lab, part of the 2019 Cross-Language Evaluation Forum (CLEF). CheckThat! proposes two complementary tasks. Task 1: predict which claims in a political debate should be prioritized for fact-checking. Task 2: rank Web-retrieved pages against a check-worthy claim based on their usefulness for fact-checking, extract useful passages from those pages, and then use them all to decide whether the claim is factually true or false. Checkthat! provides a full evaluation framework, consisting of data in English (derived from fact-checking sources) and Arabic (gathered and annotated from scratch) and evaluation based on mean average precision (MAP) for ranking and F 1 for classification tasks.
Elsayed T., Nakov P., Barron-Cedeno A., Hasanain M., Suwaileh R., Da San Martino G., et al. (2019). CheckThat! at CLEF 2019: Automatic identification and verification of claims. Springer Verlag [10.1007/978-3-030-15719-7_41].
CheckThat! at CLEF 2019: Automatic identification and verification of claims
Barron-Cedeno A.;Da San Martino G.;
2019
Abstract
We introduce the second edition of the CheckThat! Lab, part of the 2019 Cross-Language Evaluation Forum (CLEF). CheckThat! proposes two complementary tasks. Task 1: predict which claims in a political debate should be prioritized for fact-checking. Task 2: rank Web-retrieved pages against a check-worthy claim based on their usefulness for fact-checking, extract useful passages from those pages, and then use them all to decide whether the claim is factually true or false. Checkthat! provides a full evaluation framework, consisting of data in English (derived from fact-checking sources) and Arabic (gathered and annotated from scratch) and evaluation based on mean average precision (MAP) for ranking and F 1 for classification tasks.File | Dimensione | Formato | |
---|---|---|---|
Barron-Cedeno.pdf
Open Access dal 08/04/2020
Tipo:
Postprint
Licenza:
Licenza per accesso libero gratuito
Dimensione
555.78 kB
Formato
Adobe PDF
|
555.78 kB | Adobe PDF | Visualizza/Apri |
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.