@inproceedings{elshabrawy-etal-2025-statement,
    title = "Statement-Tuning Enables Efficient Cross-lingual Generalization in Encoder-only Models",
    author = "Elshabrawy, Ahmed  and
      Nguyen, Thanh-Nhi  and
      Kang, Yeeun  and
      Feng, Lihan  and
      Jain, Annant  and
      Shaikh, Faadil Abdullah  and
      Mansurov, Jonibek  and
      Imam, Mohamed Fazli Mohamed  and
      Ortiz-Barajas, Jesus-German  and
      Chevi, Rendi  and
      Aji, Alham Fikri",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Findings of the Association for Computational Linguistics: ACL 2025",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingestion-acl-25/2025.findings-acl.835/",
    pages = "16226--16248",
    ISBN = "979-8-89176-256-5",
    abstract = "Large Language Models (LLMs) excel in zero-shot and few-shot tasks, but achieving similar performance with encoder-only models like BERT and RoBERTa has been challenging due to their architecture. However, encoders offer advantages such as lower computational and memory costs. Recent work adapts them for zero-shot generalization using Statement Tuning, which reformulates tasks into finite templates. We extend this approach to multilingual NLP, exploring whether encoders can achieve zero-shot cross-lingual generalization and serve as efficient alternatives to memory-intensive LLMs for low-resource languages. Our results show that state-of-the-art encoder models generalize well across languages, rivaling multilingual LLMs while being more efficient. We also analyze multilingual Statement Tuning dataset design, efficiency gains, and language-specific generalization, contributing to more inclusive and resource-efficient NLP models. We release our code and models."
}