@article{torunoglu-selamet-etal-2026-parallel,
    title = "A Parallel Cross-Lingual Benchmark for Multimodal Idiomaticity Understanding",
    author = {Toruno{\u{g}}lu-Selamet, Dilara  and
      Arslan, Do{\u{g}}ukan  and
      Wilkens, Rodrigo  and
      He, Wei  and
      Eryi{\u{g}}it, Doruk  and
      Pickard, Thomas  and
      Pagano, Adriana S.  and
      Villavicencio, Aline  and
      Eryi{\u{g}}it, G{\"u}l{\c{s}}en  and
      Abuczki, {\'A}gnes  and
      Cardoso, Aida  and
      Lazarenka, Alesia  and
      Almassova, Dina  and
      Mendes, Am{\'a}lia  and
      Kanellopoulou, Anna  and
      Brosa-Rodriguez, Antoni  and
      Valkovska, Baiba  and
      Wojtowicz, Beata  and
      Pedersen, Bolette  and
      Hidalgo-Ternero, Carlos Manuel  and
      Liebeskind, Chaya  and
      Joki{\'c}, Danka  and
      Alves, Diego  and
      Triantafyllidi, Eleni  and
      Velldal, Erik  and
      Philippy, Fred  and
      Valunaite Oleskeviciene, Giedre  and
      Rizgeliene, Ieva  and
      Skadina, Inguna  and
      Lobzhanidze, Irina  and
      Haugen, Isabell Stinessen  and
      Krito, Jauza Akbar  and
      Markovi{\'c}, Jelena M.  and
      Monti, Johanna  and
      Sauca, Josue Alejandro  and
      Dobrovoljc Zor, Kaja  and
      Ugwuanyi, Kingsley O.  and
      Rituma, Laura  and
      {\O}vrelid, Lilja  and
      Agro, Maha Tufail  and
      Abjalova, Manzura  and
      Chatzigrigoriou, Maria  and
      S{\'a}nchez Ramos, Mar{\'i}a del Mar  and
      Pendevska, Marija  and
      Seyyedrezaei, Masoumeh  and
      Shamsfard, Mehrnoush  and
      Ahsan, Momina  and
      Khan, Muhammad Ahsan Riaz  and
      Norman, Nathalie Carmen Hau  and
      Erdem Ayy{\i}ld{\i}z, Nilay  and
      Hosseini-Kivanani, Nina  and
      Ligeti-Nagy, No{\'e}mi  and
      Naeem, Numaan  and
      Kanishcheva, Olha  and
      Yatsyshyna, Olha  and
      Orel, Daniil  and
      Giommarelli, Petra  and
      Osenova, Petya  and
      Garabik, Radovan  and
      Semou, Regina E.  and
      Rebechi, Rozane  and
      Pranida, Salsabila Zahirah  and
      Touileb, Samia  and
      Nimb, Sanni  and
      Ahmad, Sarfraz  and
      Sharipova, Sarvinoz  and
      Golan, Shahar  and
      Ji, Shaoxiong  and
      Aboh, Sopuruchi Christian  and
      Sucur, Srdjan  and
      Markantonatou, Stella  and
      Olsen, Sussi  and
      Tajalli, Vahide  and
      Lipp, Veronika  and
      Giouli, Voula  and
      Ye{\c{s}}ildal Erayd{\i}n, Yelda  and
      Saaberi, Zahra  and
      Xie, Zhuohan},
    editor = "Piperidis, Stelios  and
      Bel, N{\'u}ria  and
      van den Heuvel, Henk  and
      Ide, Nancy  and
      Krek, Simon  and
      Toral, Antonio",
    journal = "International Conference on Language Resources and Evaluation",
    volume = "main",
    month = may,
    year = "2026",
    address = "Palma de Mallorca, Spain",
    publisher = "ELRA Language Resource Association",
    url = "https://preview.aclanthology.org/ingest-lrec/2026.lrec-main.741/",
    pages = "9434--9448",
    abstract = "Potentially idiomatic expressions (PIEs) carry meanings inherently tied to the everyday experience of a given language community. As such, they constitute an interesting challenge for assessing the linguistic (and to some extent cultural) capabilities of NLP systems. In this paper, we present XMPIE, a parallel multilingual and multimodal dataset of potentially idiomatic expressions. The dataset, containing 34 languages and over ten thousand items, allows comparative analyses of idiomatic patterns among language-specific realisations and preferences in order to gather insights about shared cultural aspects. This parallel dataset allows evaluation of language model performance for a given PIE in different languages and whether idiomatic understanding in one language can be transferred to another. Moreover, the dataset supports the study of PIEs across textual and visual modalities, to measure to what extent PIE understanding in one modality transfers or implies in understanding in another modality (text vs. image). The data was created by language experts, with both textual and visual components crafted under multilingual guidelines, and each PIE is accompanied by five images representing a spectrum from idiomatic to literal meanings, including semantically related and random distractors. The result is a high-quality benchmark for evaluating multilingual and multimodal idiomatic language understanding."
}