Jingyuan Tian
2026
SDC-LoRA: Singular-Subspace Drift Controlled LoRA to Mitigate Knowledge Forgetting
Geyuan Zhang | Xiaofei Zhou | Shihao Liu | Jingyuan Tian | Jizheng Ma
Findings of the Association for Computational Linguistics: ACL 2026
Geyuan Zhang | Xiaofei Zhou | Shihao Liu | Jingyuan Tian | Jizheng Ma
Findings of the Association for Computational Linguistics: ACL 2026
Knowledge forgetting is a central challenge when adapting LLMs to new tasks. Prior studies indicate that pretrained knowledge is concentrated in the principal singular subspace of pretrained weight W0; so recent Low-Rank Adaptation (LoRA) variants initialize LoRA in the minor subspace to steer early updates away from principal directions and mitigate forgetting. However, we observe that during fine-tuning, the update direction progressively shifts from the minor to the principal subspace, which is called as Singular-subspace Drift (SD), thereby allocating more energy to the directions that carry pretrained knowledge and leaving a persistent risk of forgetting. To address this issue, we propose Singular-subspace Drift Controlled LoRA (SDC-LoRA), which constrains the growth of update energy in the principal singular subspace of W0 and thus mitigate SD. SDC-LoRA proposes Principal Subspace Energy-Controlled Learning, using Spectral Calibration factor 𝛾sc to selectively downscale gradients along the principal singular subspace of W0 while keeping minor-subspace updates unchanged. Across extensive experiments with LLaMA-3.1-8B-Instruct and Qwen2.5-7B-Chat on MetaMathQA and CodeFeedback, SDC-LoRA mitigates forgetting on MMLU, TruthfulQA, and HellaSwag while matching or improving GSM8K and HumanEval, offering a practical route to adapt LLMs without sacrificing prior knowledge.
CFlowPsyD: An Analysis-Enhanced Dataset for Asynchronous Psychological Counseling through Self-Optimizing Multi-Agent Framework
Donghao Li | Yifan Deng | Jinta Weng | Xingsheng Zhang | Chenxu Niu | Jingyuan Tian | Heyan Huang | Yue Hu
Findings of the Association for Computational Linguistics: ACL 2026
Donghao Li | Yifan Deng | Jinta Weng | Xingsheng Zhang | Chenxu Niu | Jingyuan Tian | Heyan Huang | Yue Hu
Findings of the Association for Computational Linguistics: ACL 2026
Asynchronous psychological counseling (APC) represents a crucial mental health service modality that transcends temporal and spatial constraints. However, its development faces significant data scarcity challenges: due to stringent privacy protection requirements and professional ethical considerations, direct collection of conversational data from authentic APC scenarios is virtually impossible. To address this challenge, we design a self-optimizing multi-agent framework for counseling dialogue generation, CFlowPsy, which utilizes a small amount of real anonymized counseling cases as seed data to synthesize diverse problem-solving-oriented APC conversations through large language models. Specifically, the framework employs a Persona-Flow module to continuously track and update clients’ basic information, real-time emotions, and counseling progress, providing dynamic personalized analytical support for counselors and enabling self-optimization of generated dialogues. Simultaneously, the framework ensures that generated interventions contain explicit reasoning processes, demonstrating clear psychological analysis and logic, thereby enhancing the accuracy and consistency of responses. Under this framework, we develop the first Chinese APC dataset, CFlowPsyD, comprising 1,700 high-quality extended conversations. Extensive experiments and human evaluations confirm that the proposed CFlowPsyD dataset successfully simulates human-like APC processes.