Voight-Kampff 2027: Can Your Model Write Like a Human?

Large Language Models (LLMs) can produce remarkably fluent text, but how human-like is it when tested by independent detection systems?

The Voight-Kampff task invites participants to generate texts that challenge human-vs-machine classifiers. The task is organised as a builder-breaker challenge in collaboration with the PAN Lab. ELOQUENT participants generate texts, while PAN participants build systems that try to detect them. This collaboration has been part of the task since its introduction.

We ask a straightforward question:

Whether you have expertise in prompt engineering, open-weight models, commercial LLMs, fine-tuning, decoding strategies, or a completely new idea, you are welcome to participate.

What is the challenge?

You will receive a set of descriptions derived from human-authored texts. Each item contains information about the content to be covered and may also specify its genre and style.

Previous editions have included genres such as newswire, encyclopedia-style writing, biographies, fan fiction, reports, and transcripts.

Your goal is to use a generative system to turn each description into natural text of approximately 500 words.

The challenge is not simply to generate fluent text. Rather, it is to generate text that is difficult to distinguish from genuine human writing.

How does this task work?

1. Download the task data. The 2027 data will be released soon.
2. Generate one text for each test item.
3. Submit your generated texts using the required file structure.
4. We challenge the PAN detectors. Your outputs will be evaluated using human-vs-machine classifiers developed in the corresponding PAN task.
5. See how human your system appears. Results will indicate how successfully your outputs challenge independent detection systems.

What approaches can I try?

Based on previous editions and the broader AI-text detection literature, participants may explore approaches such as:

1. Prompt engineering or multi-stage prompting (Gunti et al., 2026).
2. Comparing different foundation models (Ji et al., 2025; Wang et al., 2026).
3. Open-weight or proprietary LLMs (Ji et al., 2025; Wang et al., 2026).
4. Fine-tuning or model adaptation (Ullah et al., 2026).
5. Decoding and sampling strategies (Gunti et al., 2026).
6. Retrieval or source grounding (Pagan et al., 2025).
7. Style or genre conditioning (Markowitz et al., 2024).
8. Candidate generation and selection (Gunti et al., 2026).
9. Detector-aware generation (Gunti et al., 2026).
10. Human-in-the-loop editing (Ji et al., 2025; Palmer, 2024).
11. Multilingual or cross-lingual strategies (Gunti et al., 2026).
12. Or a new method that we have not considered.

Complicated approaches are optional. Previous editions have included approaches ranging from straightforward prompting to controlled decoding, detector-guided optimisation, model adaptation, and human-in-the-loop strategies.

What matters is that you clearly describe what you did so that the community can understand which generation choices make machine-generated writing harder — or easier — to detect.

Task data

The 2027 task data will be published soon.

For reference, previous Voight-Kampff data are available on Hugging Face:

ELOQUENT Voight-Kampff dataset on Hugging Face

Sample from the 2024 data

{
  "voight-kampfftesttopics": {
    "language": "en",
    "date": "2024",
    "type": "example",
    "source": "eloquent organisers",
    "prompt": "Write a text of about 500 words which covers the following items: ",
    "topics": [
      {
        "id": "001",
        "Genre and Style": "Encyclopedia",
        "Content": [
          "Uralic languages descended from Proto-Uralic language from 7,000 to 10,000 years ago.",
          "Uralic languages spoken by 25 million people in northeastern Europe, northern Asia, and North America.",
          "Hungarian, Estonian, and Finnish are the most important Uralic languages.",
          "Attempts to trace genealogy of Uralic languages to earlier periods have been hampered by lack of evidence.",
          "Uralic and Indo-European languages are not thought to be related, but speculation exists.",
          "Uralic languages consist of two groups: Finno-Ugric and Samoyedic.",
          "Finno-Ugric and Samoyedic have given rise to divergent subgroups of languages.",
          "Degree of similarity in Finno-Ugric languages is comparable to that between English and Russian.",
          "Finnish and Estonian, closely related members of Finno-Ugric, differ similarly to diverse dialects of the same language."
        ]
      }
    ]
  }
}

Submission

One test item = one text file

Submit a .zip archive containing a directory named after your team and run, with exactly one plain-text (.txt) file for every test item.

For example, if the test set contains items 053 through 072, the correct structure is:

OurTeamName_Run1/
    053.txt
    054.txt
    055.txt
    056.txt
    ...
    071.txt
    072.txt

Important: do not combine test items

OurTeamName_Run1/
    all_outputs.txt

Use the exact test-item IDs

Each filename must correspond exactly to the supplied test-item identifier.

For example:

053.txt
054.txt
055.txt

Do not rename individual test items using participant names, model names, descriptions, or repeated identifiers.

The combination of the team/run directory name + test-item ID must uniquely identify every submitted text.

Correct:

OurTeam_Run1/053.txt
OurTeam_Run1/054.txt

Duplicated or missing identifiers can cause texts to be merged, overwritten, or omitted when results from multiple PAN systems are combined.

Multiple runs

You are welcome to submit multiple experimental conditions. However, each run must be clearly separated and uniquely named.

For example:

OurTeam_Baseline/
    053.txt
    ...
    072.txt

OurTeam_StylePrompt/
    053.txt
    ...
    072.txt

OurTeam_Finetuned/
    053.txt
    ...
    072.txt

Please do not mix outputs from different experimental conditions in the same directory unless they belong to the same declared run.

Before submitting: check your archive

Before uploading your submission, please confirm that:

• There is one .txt file for every required test item.
• Every expected item ID appears exactly once in each run.
• No test items are missing.
• No IDs are duplicated.
• No file contains responses for multiple test items.
• Each run has a unique directory/run name.
• The filenames match the supplied task IDs exactly.
• Files contain only the generated text for that item, without JSON wrappers, detector scores, labels, or other metadata.
• Text files are preferably saved as UTF-8 plain text.
• The .zip archive opens successfully after compression.

For example, if the task contains 20 test items, each submitted run should contain 20 individual .txt files.

Submission delivery

Upload the .zip archive through the official submission location:

Submission form: Will be available soon.

After submission, please retain an unchanged copy of the exact .zip archive that you uploaded.

Where possible, evaluation runs should be exchanged using the designated shared download location rather than relying only on email attachments. Institutional email systems may block or alter attachments through S/MIME or other security filtering.

If an organiser reports that an attachment cannot be opened, please provide the same original archive through the designated shared download location rather than recompressing or restructuring the submission.

Submission metadata

Along with each run, participants should briefly report:

• Team name.
• Unique run name.
• Model or models used.
• Prompt or generation strategy.
• Additional training or fine-tuning, if any.
• Decoding or post-processing modifications, if any.
• External data or retrieval sources, if any.
• Human intervention, if any.

A full methodological description can be provided in the accompanying working-notes paper.

Why participate?

Voight-Kampff is not only a leaderboard challenge. It is an opportunity to investigate a rapidly evolving scientific question:

Your submission can help us understand:

• Which linguistic signals current detectors rely on.
• Which signals modern LLMs still struggle to reproduce.
• Whether detector weaknesses generalise across models and genres.
• Whether sophisticated prompting actually improves human-likeness.
• How much generation strategy matters compared with model choice.
• Which texts are naturally easier or harder to humanise.
• Whether today's strongest generated texts are becoming operationally indistinguishable from genuine human writing.

The task is suitable both for teams interested in text generation and researchers interested in AI detection, stylometry, adversarial NLP, computational linguistics, evaluation, and human-AI communication.

Students, academic researchers, industry practitioners, and independent teams are all encouraged to participate.

New to the task?

That is completely fine.

You do not need to reproduce a previous winning system or begin with a complicated pipeline.

A straightforward baseline using your preferred language model and the supplied prompts is enough to enter. From there, you can experiment with prompting, style, model choice, decoding, or any other idea you would like to test.

The diversity of approaches is part of the experiment.

Simple ideas are welcome. Unexpected ideas are especially welcome.

Important dates

The final CLEF 2027 timeline will be confirmed soon.

• Pre-announcement: Coming soon.
• 2027 test data: Planned for late January.
• Participant run submission: To be confirmed.
• Results: Planned for mid-May.
• Participant papers: Planned for the end of May.
• Participant-paper reviews: Planned for approximately mid-summer.
• Final reporting: Planned for the end of June.

Please check this page for confirmed dates as the task schedule is finalised.

References