Mitigating Language Bias in Multilingual Sentence Embeddings for Cross-Lingual Similarity Estimation

Kanade Nonomura, Keita Fukushima, Risa Kondo, Tomoyuki Kajiwara


Abstract
We disentangle multilingual sentence embeddings into language-dependent and language-agnostic components, leveraging the latter to improve cross-lingual similarity estimation. Previous studies on this approach have trained disentanglers by combining intra-component constraints, which either align or disalign language-dependent embeddings or language-agnostic embeddings, with inter-component constraints across both embeddings. However, when and how these constraints are effective remains unclear. Our experiments on sentence similarity estimation and machine translation quality estimation revealed that while intra-component constraints and the combination of both constraints are effective for encoder-based multilingual sentence embeddings, inter-component constraints are effective for decoder-based ones. Furthermore, our detailed analysis revealed distinct roles: intra-component constraints improve uniformity within the embedding space, while inter-component constraints enhance cross-lingual alignment between parallel sentences.
Anthology ID:
2026.starsem-conference.26
Volume:
Proceedings of the 15th Joint Conference on Lexical and Computational Semantics (*SEM 2026)
Month:
July
Year:
2026
Address:
San Diego, California, United States
Editors:
Saif M. Mohammad, Nedjma Ousidhoum
Venues:
*SEM | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
385–394
Language:
URL:
https://preview.aclanthology.org/ingest-acl-workshops/2026.starsem-conference.26/
DOI:
Bibkey:
Cite (ACL):
Kanade Nonomura, Keita Fukushima, Risa Kondo, and Tomoyuki Kajiwara. 2026. Mitigating Language Bias in Multilingual Sentence Embeddings for Cross-Lingual Similarity Estimation. In Proceedings of the 15th Joint Conference on Lexical and Computational Semantics (*SEM 2026), pages 385–394, San Diego, California, United States. Association for Computational Linguistics.
Cite (Informal):
Mitigating Language Bias in Multilingual Sentence Embeddings for Cross-Lingual Similarity Estimation (Nonomura et al., *SEM 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl-workshops/2026.starsem-conference.26.pdf