Abhinav Mishra
2025
ANVITA : A Multi-pronged Approach for Enhancing Machine Translation of Extremely Low-Resource Indian Languages
Sivabhavani J
|
Daneshwari Kankanwadi
|
Abhinav Mishra
|
Biswajit Paul
Proceedings of the Tenth Conference on Machine Translation
India has a rich diverse linguistic landscape including 22 official languages and 122 major languages. Most of these 122 languages fall into low, extremely low resource categories and pose significant challenges in building robust machine translation system. This paper presents ANVITA Indic LR machine translation system submitted to WMT 2025 shared task on Low-Resource Indic Language Translation covering three extremely low-resource Indian languages Nyshi, Khasi, and Kokborok. A transfer learning based strategy is adopted and selected suitable public pretrained models (NLLB, ByT5), considering aspects such as language, script, tokenization and fine-tuned with the organizer provided dataset. Further, to tackle low-resource language menace better, the pretrained models are enriched with new vocabulary for improved representation of these three languages and selectively augmented data with related-language corpora, supplied by the organizer. The contrastive submissions however made use of supplementary corpora sourced from the web, generated synthetically, and drawn from proprietary data. On the WMT 2025 official test set, ANVITA achieved BLEU score of 2.41-11.59 with 2.2K to 60K corpora and 6.99-19.43 BLUE scores with augmented corpora. Overall ANVITA ranked first for {Nyishi, Kokborok}↔English and second for Khasi↔English across evaluation metrics including BLUE, METEOR, ROUGE-L, chrF and TER.
2022
WebCrawl African : A Multilingual Parallel Corpora for African Languages
Pavanpankaj Vegi
|
Sivabhavani J
|
Biswajit Paul
|
Abhinav Mishra
|
Prashant Banjare
|
Prasanna K R
|
Chitra Viswanathan
Proceedings of the Seventh Conference on Machine Translation (WMT)
WebCrawl African is a mixed domain multilingual parallel corpora for a pool of African languages compiled by ANVITA machine translation team of Centre for Artificial Intelligence and Robotics Lab, primarily for accelerating research on low-resource and extremely low-resource machine translation and is part of the submission to WMT 2022 shared task on Large-Scale Machine Translation Evaluation for African Languages under the data track. The corpora is compiled through web data mining and comprises 695K parallel sentences spanning 74 different language pairs from English and 15 African languages, many of which fall under low and extremely low resource categories. As a measure of corpora usefulness, a MNMT model for 24 African languages to English is trained by combining WebCrawl African corpora with existing corpus and evaluation on FLORES200 shows that inclusion of WebCrawl African corpora could improve BLEU score by 0.01-1.66 for 12 out of 15 African to English translation directions and even by 0.18-0.68 for the 4 out of 9 African to English translation directions which are not part of WebCrawl African corpora. WebCrawl African corpora includes more parallel sentences for many language pairs in comparison to OPUS public repository. This data description paper captures creation of corpora and results obtained along with datasheets. The WebCrawl African corpora is hosted on github repository.
Search
Fix author
Co-authors
- Sivabhavani J 2
- Biswajit Paul 2
- Prashant Banjare 1
- Prasanna K R 1
- Daneshwari Kankanwadi 1
- show all...
Venues
- wmt2