@inproceedings{hamidullah-etal-2025-sonar,
    title = "{SONAR}-{SLT}: Multilingual Sign Language Translation via Language-Agnostic Sentence Embedding Supervision",
    author = "Hamidullah, Yasser  and
      Yazdani, Shakib  and
      Oguz, Cennet  and
      Van Genabith, Josef  and
      Espa{\~n}a-Bonet, Cristina",
    editor = "Haddow, Barry  and
      Kocmi, Tom  and
      Koehn, Philipp  and
      Monz, Christof",
    booktitle = "Proceedings of the Tenth Conference on Machine Translation",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/author-page-yu-wang-polytechnic/2025.wmt-1.18/",
    doi = "10.18653/v1/2025.wmt-1.18",
    pages = "301--313",
    ISBN = "979-8-89176-341-8",
    abstract = "Sign language translation (SLT) is typically trained with text in a single spoken language, which limits scalability and cross-language generalization. Earlier approaches have replaced gloss supervision with text-based sentence embeddings, but up to now, these remain tied to a specific language and modality. In contrast, here we employ language-agnostic, multimodal embeddings trained on text and speech from multiple languages to supervise SLT, enabling direct multilingual translation. To address data scarcity, we propose a coupled augmentation method that combines multilingual target augmentations (i.e. translations into many languages) with video-level perturbations, improving model robustness. Experiments show consistent BLEURT gains over text-only sentence embedding supervision, with larger improvements in low-resource settings. Our results demonstrate that language-agnostic embedding supervision, combined with coupled augmentation, provides a scalable and semantically robust alternative to traditional SLT training."
}