Mitigating Language Bias in Multilingual Sentence Embeddings for Cross-Lingual Similarity Estimation
Kanade Nonomura, Keita Fukushima, Risa Kondo, Tomoyuki Kajiwara
Abstract
We disentangle multilingual sentence embeddings into language-dependent and language-agnostic components, leveraging the latter to improve cross-lingual similarity estimation. Previous studies on this approach have trained disentanglers by combining intra-component constraints, which either align or disalign language-dependent embeddings or language-agnostic embeddings, with inter-component constraints across both embeddings. However, when and how these constraints are effective remains unclear. Our experiments on sentence similarity estimation and machine translation quality estimation revealed that while intra-component constraints and the combination of both constraints are effective for encoder-based multilingual sentence embeddings, inter-component constraints are effective for decoder-based ones. Furthermore, our detailed analysis revealed distinct roles: intra-component constraints improve uniformity within the embedding space, while inter-component constraints enhance cross-lingual alignment between parallel sentences.- Anthology ID:
- 2026.starsem-conference.26
- Volume:
- Proceedings of the 15th Joint Conference on Lexical and Computational Semantics (*SEM 2026)
- Month:
- July
- Year:
- 2026
- Address:
- San Diego, California, United States
- Editors:
- Saif M. Mohammad, Nedjma Ousidhoum
- Venues:
- *SEM | WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 385–394
- Language:
- URL:
- https://preview.aclanthology.org/ingest-acl-workshops/2026.starsem-conference.26/
- DOI:
- Cite (ACL):
- Kanade Nonomura, Keita Fukushima, Risa Kondo, and Tomoyuki Kajiwara. 2026. Mitigating Language Bias in Multilingual Sentence Embeddings for Cross-Lingual Similarity Estimation. In Proceedings of the 15th Joint Conference on Lexical and Computational Semantics (*SEM 2026), pages 385–394, San Diego, California, United States. Association for Computational Linguistics.
- Cite (Informal):
- Mitigating Language Bias in Multilingual Sentence Embeddings for Cross-Lingual Similarity Estimation (Nonomura et al., *SEM 2026)
- PDF:
- https://preview.aclanthology.org/ingest-acl-workshops/2026.starsem-conference.26.pdf