@inproceedings{nakai-etal-2025-treplina,
    title = "{TR}ep{L}i{N}a: Layer-wise {CKA}+{REPINA} Alignment Improves Low-Resource Machine Translation in Aya-23 8{B}",
    author = "Nakai, Toshiki  and
      Chikkala, Ravikiran  and
      Oberkircher, Lena  and
      Jennings, Nicholas  and
      Skachkova, Natalia  and
      Anikina, Tatiana  and
      Alabi, Jesujoba Oluwadara",
    editor = "Shukla, Ankita  and
      Kumar, Sandeep  and
      Bedi, Amrit Singh  and
      Chakraborty, Tanmoy",
    booktitle = "Proceedings of the 1st Workshop on Multimodal Models for Low-Resource Contexts and Social Impact (MMLoSo 2025)",
    month = dec,
    year = "2025",
    address = "Mumbai, India",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-ijcnlp-aacl/2025.mmloso-1.3/",
    pages = "25--34",
    ISBN = "979-8-89176-311-1",
    abstract = "The 2025 Multimodal Models for Low-Resource Contexts and Social Impact (MMLoSo) Language Challenge addresses one of India{'}s most pressing linguistic gaps: the lack of resources for its diverse low-resource languages (LRLs). In this study, we investigate whether enforcing cross-lingual similarity in specific internal layers of a decoder-only multilingual large language model (LLM) can improve translation quality from LRL to high-resource language (HRL). Specifically, we combine Centered Kernel Alignment (CKA), a similarity metric that encourages representations of different languages to align, with REPINA, a regularization method that constrains parameter updates to remain close to the pretrained model, into a joint method we call TRepLiNa. In this research project, we experiment with zero-shot, few-shot, and fine-tuning settings using Aya-23 8B with QLoRA across MMLoSo shared task language pairs (Mundari, Santali, Bhili) with Hindi/English pivots. Our results show that aligning mid-level layers using TRepLiNa (CKA+REPINA) is a low-cost, practical approach to improving LRL translation, especially in data-scarce settings. Upon acceptance of the paper, we make our code public."
}