Does your LLM know what it is talking about?
This task was defined to test and verify that a system based on a generative language model is able to handle material from some given topical domain of interest, by having systems automatically generate and respond to quiz tests on selected topics.
Can it be trusted? Or does it make stuff up?
This task was defined test whether your model is able to detect hallucinations in both human-authored and machine-generated contexts.
Will it respond with the same content to all of us?
This task was defined to test the capability of a model to handle input variation -- e.g. dialectal, sociolectal, and cross-cultural -- as represented by human-generated equivalent but non-identical varieties of input prompts.
More information on the task page!
Has a machine written this? Or has a human author put together these words?
This task was defined to explore whether automatically-generated text can be distinguished from human-authored text, and was organised in collaboration with the PAN lab at CLEF.
The first ELOQUENT Workshop in Grenoble, September 9-12 2024:
The CLEF program has three ELOQUENT events:
- Wednesday, September 11, 11:15-12:45: a brief plenary overview of the year's experiments;
- Thursday, September 12, 11:15-12:45: a participant workshop on results and learnings from the 2024 tasks;
- Thursday, September 12, 14:00-15:30: a forward-looking workshop on the 2025 tasks. and, in addition, the PAN program has a section on the Voight-Kampff task.
- Fall 2023: discussion and task formulation
- February 2024: tasks open and public announcement of tasks on mailing lists
- Last week of March: ECIR presentation of ELOQUENT
- 22 April 2024: registration for participation closes
- May 2024: submission deadline of experimental runs from participants
- June 2024: participant report submission deadline
- July 2024: camera ready report submission deadline
- September 2024: workshop at CLEF
- Silo AI, Helsinki: Jussi Karlgren
- Accenture and University of Helsinki: Aarne Talman
- RISE ICT, Stockholm : Luise Dürlich, Evangelia Gogoulou, Joakim Nivre
- University of Edinburgh: Liane Guillou
- AI Sweden, Stockholm: Magnus Sahlgren
CLEF is an annual conference which collects evaluation experiments and shared tasks on a broad range of information systems. More information on the CLEF page!
ELOQUENT will run again next year, with a workshop to be organised at CLEF in Madrid in September 2025.
ELOQUENT will run again next year, with a workshop to be organised at CLEF in Madrid in September 2025.