Jingyuan Tian


2026

Knowledge forgetting is a central challenge when adapting LLMs to new tasks. Prior studies indicate that pretrained knowledge is concentrated in the principal singular subspace of pretrained weight W0; so recent Low-Rank Adaptation (LoRA) variants initialize LoRA in the minor subspace to steer early updates away from principal directions and mitigate forgetting. However, we observe that during fine-tuning, the update direction progressively shifts from the minor to the principal subspace, which is called as Singular-subspace Drift (SD), thereby allocating more energy to the directions that carry pretrained knowledge and leaving a persistent risk of forgetting. To address this issue, we propose Singular-subspace Drift Controlled LoRA (SDC-LoRA), which constrains the growth of update energy in the principal singular subspace of W0 and thus mitigate SD. SDC-LoRA proposes Principal Subspace Energy-Controlled Learning, using Spectral Calibration factor 𝛾sc to selectively downscale gradients along the principal singular subspace of W0 while keeping minor-subspace updates unchanged. Across extensive experiments with LLaMA-3.1-8B-Instruct and Qwen2.5-7B-Chat on MetaMathQA and CodeFeedback, SDC-LoRA mitigates forgetting on MMLU, TruthfulQA, and HellaSwag while matching or improving GSM8K and HumanEval, offering a practical route to adapt LLMs without sacrificing prior knowledge.
Asynchronous psychological counseling (APC) represents a crucial mental health service modality that transcends temporal and spatial constraints. However, its development faces significant data scarcity challenges: due to stringent privacy protection requirements and professional ethical considerations, direct collection of conversational data from authentic APC scenarios is virtually impossible. To address this challenge, we design a self-optimizing multi-agent framework for counseling dialogue generation, CFlowPsy, which utilizes a small amount of real anonymized counseling cases as seed data to synthesize diverse problem-solving-oriented APC conversations through large language models. Specifically, the framework employs a Persona-Flow module to continuously track and update clients’ basic information, real-time emotions, and counseling progress, providing dynamic personalized analytical support for counselors and enabling self-optimization of generated dialogues. Simultaneously, the framework ensures that generated interventions contain explicit reasoning processes, demonstrating clear psychological analysis and logic, thereby enhancing the accuracy and consistency of responses. Under this framework, we develop the first Chinese APC dataset, CFlowPsyD, comprising 1,700 high-quality extended conversations. Extensive experiments and human evaluations confirm that the proposed CFlowPsyD dataset successfully simulates human-like APC processes.