@inproceedings{kim-etal-2026-gaia,
    title = "{GAIA}-v2-{LILT}: Multilingual Adaptation of Agent Benchmark beyond Translation",
    author = "Kim, Yunsu  and
      Uhlig, Kaden  and
      Wuebker, Joern",
    editor = "Huang, Kaiyu  and
      Mo, Fengran  and
      Chen, Pinzhen  and
      Jiang, Meng",
    booktitle = "Proceedings of the 1st Workshop on Multilinguality in the Era of Large Language Models ({M}e{LLM} 2026)",
    month = jul,
    year = "2026",
    address = "San Diego, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl-workshops/2026.mellm-1.13/",
    pages = "140--148",
    ISBN = "979-8-89176-430-9",
    abstract = "Agent benchmarks remain largely English-centric, while their multilingual versions are often built with machine translation (MT) and limited post-editing. We argue that, for agentic tasks, this minimal workflow can easily break benchmark validity through query-answer misalignment or culturally off-target context. We propose a refined workflow for adapting English benchmarks into multiple languages with explicit functional alignment, cultural alignment, and difficulty calibration using both automated checks and human review. Using this workflow, we introduce GAIA-v2-LILT, a re-audited multilingual extension of GAIA covering five non-English languages. In experiments, our workflow improves agent success rates by up to 32.7{\%} over minimally translated versions, bringing the closest audited setting to within 3.1{\%} of English performance while substantial gaps remain in many other cases. This indicates that a substantial share of the multilingual performance gap is benchmark-induced measurement error, motivating task-level alignment when adapting English benchmarks across languages. The data is available as part of the MAPS package. We also release the code used in our experiments."
}