The Multilingual Curse at the Retrieval Layer: Evidence from Amharic

Yosef Worku Alemneh, Kidist Amde Mekonnen, Maarten de Rijke


Abstract
Multilingual retrieval increasingly underpins cross-lingual question answering and retrieval-augmented generation. Strong zero-shot scores on multilingual benchmarks are often taken as evidence that current encoders transfer reliably across many languages. We argue that this assumption breaks down for underrepresented, morphologically rich languages, and use Amharic as a diagnostic case. Under a shared passage retrieval protocol covering dense, late-interaction, learned sparse, and cross-encoder paradigms, we compare zero-shot multilingual retrievers, Amharic-fine-tuned multilingual retrievers, and monolingual Amharic retrievers. The strongest zero-shot multilingual retriever underperforms the strongest monolingual Amharic first-stage retriever by 23% relative MRR@10. Fine-tuning two recent multilingual embedding models on the same Amharic supervision yields 32–60% relative MRR@10 gains over zero-shot, but the best Amharic-fine-tuned multilingual model remains below the strongest monolingual Amharic retriever. These findings indicate that zero-shot multilingual retrieval is not a sufficient proxy for equitable information access in the LLM era: for underrepresented languages, retrieval must be evaluated and adapted in language rather than inferred from aggregate multilingual benchmarks. To foster future research, we publicly release our trained models, dataset, and codebase at https://github.com/rasyosef/amharic-neural-ir.
Anthology ID:
2026.mellm-1.19
Volume:
Proceedings of the 1st Workshop on Multilinguality in the Era of Large Language Models (MeLLM 2026)
Month:
July
Year:
2026
Address:
San Diego, United States
Editors:
Kaiyu Huang, Fengran Mo, Pinzhen Chen, Meng Jiang
Venues:
MeLLM | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
201–210
Language:
URL:
https://preview.aclanthology.org/ingest-acl-workshops/2026.mellm-1.19/
DOI:
Bibkey:
Cite (ACL):
Yosef Worku Alemneh, Kidist Amde Mekonnen, and Maarten de Rijke. 2026. The Multilingual Curse at the Retrieval Layer: Evidence from Amharic. In Proceedings of the 1st Workshop on Multilinguality in the Era of Large Language Models (MeLLM 2026), pages 201–210, San Diego, United States. Association for Computational Linguistics.
Cite (Informal):
The Multilingual Curse at the Retrieval Layer: Evidence from Amharic (Alemneh et al., MeLLM 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl-workshops/2026.mellm-1.19.pdf