Layer-aware Dual-directional Modulation for Low-resource Machine Translation
Siqi Zhang, Ran Song, Shuting Jiang, Yuxin Huang, Zhengtao Yu
Abstract
Although Large Language Models (LLMs) have achieved remarkable success in Machine Translation (MT), a significant performance gap persists between high-resource and low-resource languages due to imbalanced pre-training data. In this paper, we first investigate the internal mechanisms driving this performance disparity from a layer-wise perspective.We propose a metric termed Activation Disparity (𝛥 R) to quantify the activation divergence between high- and low-resource MT. Based on this metric, we distinguish between Task-Adaptive Layers (TAL, 𝛥 R > 0) that encode task-specific signals and Legacy-Inert Layers (LIL, 𝛥 R < 0) dominated by pre-trained bias. Leveraging this finding, we propose the Layer-aware Dual-directional Modulation (LaDM). Integrated with Low-Rank Adaptation (LoRA), LaDM employs a sparse strategy to bidirectionally modulate optimization dynamics. Specifically, it amplifies contributions from TAL to accelerate feature consolidation while inhibiting LIL to dampen misaligned legacy biases. Extensive experiments on Chinese-to-seven low-resource language translation using Llama-3.1, Qwen2.5, and Gemma-2 demonstrate that LaDM significantly outperforms standard LoRA fine-tuning, achieving an average improvement of 1.73 spBLEU.Code is available at https://github.com/zzssqqq/LaDM.- Anthology ID:
- 2026.findings-acl.1054
- Volume:
- Findings of the Association for Computational Linguistics: ACL 2026
- Month:
- July
- Year:
- 2026
- Address:
- San Diego, California, United States
- Editors:
- Maria Liakata, Viviane P. Moreira, Jiajun Zhang, David Jurgens
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 20988–21000
- Language:
- URL:
- https://preview.aclanthology.org/ingest-acl/2026.findings-acl.1054/
- DOI:
- Cite (ACL):
- Siqi Zhang, Ran Song, Shuting Jiang, Yuxin Huang, and Zhengtao Yu. 2026. Layer-aware Dual-directional Modulation for Low-resource Machine Translation. In Findings of the Association for Computational Linguistics: ACL 2026, pages 20988–21000, San Diego, California, United States. Association for Computational Linguistics.
- Cite (Informal):
- Layer-aware Dual-directional Modulation for Low-resource Machine Translation (Zhang et al., Findings 2026)
- PDF:
- https://preview.aclanthology.org/ingest-acl/2026.findings-acl.1054.pdf