Layer-aware Dual-directional Modulation for Low-resource Machine Translation

Siqi Zhang, Ran Song, Shuting Jiang, Yuxin Huang, Zhengtao Yu


Abstract
Although Large Language Models (LLMs) have achieved remarkable success in Machine Translation (MT), a significant performance gap persists between high-resource and low-resource languages due to imbalanced pre-training data. In this paper, we first investigate the internal mechanisms driving this performance disparity from a layer-wise perspective.We propose a metric termed Activation Disparity (𝛥 R) to quantify the activation divergence between high- and low-resource MT. Based on this metric, we distinguish between Task-Adaptive Layers (TAL, 𝛥 R > 0) that encode task-specific signals and Legacy-Inert Layers (LIL, 𝛥 R < 0) dominated by pre-trained bias. Leveraging this finding, we propose the Layer-aware Dual-directional Modulation (LaDM). Integrated with Low-Rank Adaptation (LoRA), LaDM employs a sparse strategy to bidirectionally modulate optimization dynamics. Specifically, it amplifies contributions from TAL to accelerate feature consolidation while inhibiting LIL to dampen misaligned legacy biases. Extensive experiments on Chinese-to-seven low-resource language translation using Llama-3.1, Qwen2.5, and Gemma-2 demonstrate that LaDM significantly outperforms standard LoRA fine-tuning, achieving an average improvement of 1.73 spBLEU.Code is available at https://github.com/zzssqqq/LaDM.
Anthology ID:
2026.findings-acl.1054
Volume:
Findings of the Association for Computational Linguistics: ACL 2026
Month:
July
Year:
2026
Address:
San Diego, California, United States
Editors:
Maria Liakata, Viviane P. Moreira, Jiajun Zhang, David Jurgens
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
20988–21000
Language:
URL:
https://preview.aclanthology.org/ingest-acl/2026.findings-acl.1054/
DOI:
Bibkey:
Cite (ACL):
Siqi Zhang, Ran Song, Shuting Jiang, Yuxin Huang, and Zhengtao Yu. 2026. Layer-aware Dual-directional Modulation for Low-resource Machine Translation. In Findings of the Association for Computational Linguistics: ACL 2026, pages 20988–21000, San Diego, California, United States. Association for Computational Linguistics.
Cite (Informal):
Layer-aware Dual-directional Modulation for Low-resource Machine Translation (Zhang et al., Findings 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl/2026.findings-acl.1054.pdf
Checklist:
 2026.findings-acl.1054.checklist.pdf