While the quality of machine translation (MT) between widely-spoken languages has improved dramatically in recent years, training robust MT systems for languages with fewer resources remains a challenge. Endangered languages, which often lack the speaker population and written tradition needed to create text resources, are at a particular disadvantage. Developing robust MT architectures for very low-resource settings is hampered by the lack of suitable parallel corpora. To address this challenge, we introduce FormosanMT, a set of MT-ready parallel corpora for the Formosan family of endangered languages indigenous to Taiwan. Together the corpora total nearly 500,000 Formosan-Mandarin and Formosan-English sentence pairs. We share scripts for extracting these corpora from public sources, along with customizable tools for filtering, normalizing, and partitioning the data. In addition, we provide a new tokenizer for Traditional Chinese writing compatible with the popular No Language Left Behind (NLLB) MT architecture, along with updated and improved code for fine-tuning NLLB for any low-resource language pair. Finally we distribute our fully trained NLLB and OpenNMT models for the Formosan languages to and from both Mandarin and English. In addition to serving as a valuable resource for the Formosan language speaker communities, our data, code, and models will be available to NLP researchers working on endangered and low-resource language MT.

We introduce SALAN, a large-scale speech dataset covering eight of the major indigenous languages of Niger: Zarma, Hausa, Buduma, Gourmantchema, Tubu, Tamasheq, Fulfulde, and Kanuri. The final dataset exceeds 2,000 hours of audio, largely sourced from radio broadcasts and community recordings. We transcribed portions of the audio using the MMS model and conducted manual verification for 110 hours across Zarma and Hausa. We then used active learning to expand annotation to an additional 5 hours of high-uncertainty Zarma segments. To evaluate SALAN’s utility for ASR, We fine-tuned both Wav2vec2 XLS-R and Whisper on Zarma subsets and carried out additional pre-training with multilingual unlabeled data. Our best model achieved a word error rate of 25.3% and a character error rate of 6.2%. SALAN and the trained models will be made publicly available for use by researchers and speakers, with the potential to impact over 20 million individuals in Niger and neighboring countries.