Shuting Jiang
2026
Layer-aware Dual-directional Modulation for Low-resource Machine Translation
Siqi Zhang | Ran Song | Shuting Jiang | Yuxin Huang | Zhengtao Yu
Findings of the Association for Computational Linguistics: ACL 2026
Siqi Zhang | Ran Song | Shuting Jiang | Yuxin Huang | Zhengtao Yu
Findings of the Association for Computational Linguistics: ACL 2026
Although Large Language Models (LLMs) have achieved remarkable success in Machine Translation (MT), a significant performance gap persists between high-resource and low-resource languages due to imbalanced pre-training data. In this paper, we first investigate the internal mechanisms driving this performance disparity from a layer-wise perspective.We propose a metric termed Activation Disparity (𝛥 R) to quantify the activation divergence between high- and low-resource MT. Based on this metric, we distinguish between Task-Adaptive Layers (TAL, 𝛥 R > 0) that encode task-specific signals and Legacy-Inert Layers (LIL, 𝛥 R < 0) dominated by pre-trained bias. Leveraging this finding, we propose the Layer-aware Dual-directional Modulation (LaDM). Integrated with Low-Rank Adaptation (LoRA), LaDM employs a sparse strategy to bidirectionally modulate optimization dynamics. Specifically, it amplifies contributions from TAL to accelerate feature consolidation while inhibiting LIL to dampen misaligned legacy biases. Extensive experiments on Chinese-to-seven low-resource language translation using Llama-3.1, Qwen2.5, and Gemma-2 demonstrate that LaDM significantly outperforms standard LoRA fine-tuning, achieving an average improvement of 1.73 spBLEU.Code is available at https://github.com/zzssqqq/LaDM.
Breaking Consensus Bias: Unsupervised Reinforcement Learning for Machine Translation
Shuting Jiang | Ran Song | Siqi Zhang | Yuxin Huang | Shengxiang Gao | Zhengtao Yu
Findings of the Association for Computational Linguistics: ACL 2026
Shuting Jiang | Ran Song | Siqi Zhang | Yuxin Huang | Shengxiang Gao | Zhengtao Yu
Findings of the Association for Computational Linguistics: ACL 2026
Reinforcement learning (RL) excels in reasoning tasks with verifiable rewards, while its adaptation to machine translation (MT) remains challenging due to the lack of unique reward signals under multiple valid translations. Existing RL approaches for MT face either fixed references in supervised settings or the production of homogeneous references leading to mode collapse in unsupervised settings. Both limitations arise from ignoring entropy dynamics in RL-based MT. The core challenge is leveraging entropy for supervision construction and self-evolution. In this paper, we propose an Entropy-Driven Unsupervised RL for MT. Our framework integrates entropy-guided sampling for exploration, confidence-weighted label generation to transcend majority-voting bias, and uncertainty-aware optimization to prioritize high-entropy tokens. These mechanisms allow reward signals to co-evolve with model proficiency beyond fixed references. Experiments across multiple language pairs show our method outperforms supervised and unsupervised baselines by +0.63 and +2.52 average points, respectively. Our code is available at https://github.com/fortunatekiss/URLMT.
2024
Does Large Language Model Contain Task-Specific Neurons?
Ran Song | Shizhu He | Shuting Jiang | Yantuan Xian | Shengxiang Gao | Kang Liu | Zhengtao Yu
Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing
Ran Song | Shizhu He | Shuting Jiang | Yantuan Xian | Shengxiang Gao | Kang Liu | Zhengtao Yu
Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing
Large language models (LLMs) have demonstrated remarkable capabilities in comprehensively handling various types of natural language processing (NLP) tasks. However, there are significant differences in the knowledge and abilities required for different tasks. Therefore, it is important to understand whether the same LLM processes different tasks in the same way. Are there specific neurons in a LLM for different tasks? Inspired by neuroscience, this paper pioneers the exploration of whether distinct neurons are activated when a LLM handles different tasks. Compared with current research exploring the neurons of language and knowledge, task-specific neurons present a greater challenge due to their abstractness, diversity, and complexity. To address these challenges, this paper proposes a method for task-specific neuron localization based on Causal Gradient Variation with Special Tokens (CGVST). CGVST identifies task-specific neurons by concentrating on the most significant tokens during task processing, thereby eliminating redundant tokens and minimizing interference from non-essential neurons. Compared to traditional neuron localization methods, our approach can more effectively identify task-specific neurons. We conduct experiments across eight different public tasks. Experiments involving the inhibition and amplification of identified neurons demonstrate that our method can accurately locate task-specific neurons.