Shuting Jiang


2026

Although Large Language Models (LLMs) have achieved remarkable success in Machine Translation (MT), a significant performance gap persists between high-resource and low-resource languages due to imbalanced pre-training data. In this paper, we first investigate the internal mechanisms driving this performance disparity from a layer-wise perspective.We propose a metric termed Activation Disparity (𝛥 R) to quantify the activation divergence between high- and low-resource MT. Based on this metric, we distinguish between Task-Adaptive Layers (TAL, 𝛥 R > 0) that encode task-specific signals and Legacy-Inert Layers (LIL, 𝛥 R < 0) dominated by pre-trained bias. Leveraging this finding, we propose the Layer-aware Dual-directional Modulation (LaDM). Integrated with Low-Rank Adaptation (LoRA), LaDM employs a sparse strategy to bidirectionally modulate optimization dynamics. Specifically, it amplifies contributions from TAL to accelerate feature consolidation while inhibiting LIL to dampen misaligned legacy biases. Extensive experiments on Chinese-to-seven low-resource language translation using Llama-3.1, Qwen2.5, and Gemma-2 demonstrate that LaDM significantly outperforms standard LoRA fine-tuning, achieving an average improvement of 1.73 spBLEU.Code is available at https://github.com/zzssqqq/LaDM.
Reinforcement learning (RL) excels in reasoning tasks with verifiable rewards, while its adaptation to machine translation (MT) remains challenging due to the lack of unique reward signals under multiple valid translations. Existing RL approaches for MT face either fixed references in supervised settings or the production of homogeneous references leading to mode collapse in unsupervised settings. Both limitations arise from ignoring entropy dynamics in RL-based MT. The core challenge is leveraging entropy for supervision construction and self-evolution. In this paper, we propose an Entropy-Driven Unsupervised RL for MT. Our framework integrates entropy-guided sampling for exploration, confidence-weighted label generation to transcend majority-voting bias, and uncertainty-aware optimization to prioritize high-entropy tokens. These mechanisms allow reward signals to co-evolve with model proficiency beyond fixed references. Experiments across multiple language pairs show our method outperforms supervised and unsupervised baselines by +0.63 and +2.52 average points, respectively. Our code is available at https://github.com/fortunatekiss/URLMT.

2024

Large language models (LLMs) have demonstrated remarkable capabilities in comprehensively handling various types of natural language processing (NLP) tasks. However, there are significant differences in the knowledge and abilities required for different tasks. Therefore, it is important to understand whether the same LLM processes different tasks in the same way. Are there specific neurons in a LLM for different tasks? Inspired by neuroscience, this paper pioneers the exploration of whether distinct neurons are activated when a LLM handles different tasks. Compared with current research exploring the neurons of language and knowledge, task-specific neurons present a greater challenge due to their abstractness, diversity, and complexity. To address these challenges, this paper proposes a method for task-specific neuron localization based on Causal Gradient Variation with Special Tokens (CGVST). CGVST identifies task-specific neurons by concentrating on the most significant tokens during task processing, thereby eliminating redundant tokens and minimizing interference from non-essential neurons. Compared to traditional neuron localization methods, our approach can more effectively identify task-specific neurons. We conduct experiments across eight different public tasks. Experiments involving the inhibition and amplification of identified neurons demonstrate that our method can accurately locate task-specific neurons.