Jingyu Lu
2026
Are All Prompt Components Value-Neutral? Understanding the Heterogeneous Adversarial Robustness of Dissected Prompt in LLMs
Yujia Zheng | Tianhao Li | Haotian Huang | Tianyu Zeng | Jingyu Lu | Chuangxin Chu | Yuekai Huang | Ziyou Jiang | Qian Xiong | Yuyao Ge | Mingyang Li
Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
Yujia Zheng | Tianhao Li | Haotian Huang | Tianyu Zeng | Jingyu Lu | Chuangxin Chu | Yuekai Huang | Ziyou Jiang | Qian Xiong | Yuyao Ge | Mingyang Li
Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
Prompt-based adversarial attacks are a key tool for assessing the robustness of large language models (LLMs). Yet, existing studies typically treat prompts as flat text, overlooking their internal structure, different components within a prompt contribute unequally to robustness. This work introduces PromptAnatomy, a framework that decomposes prompts into functional components, and ComPerturb, a controlled perturbation method that selectively modifies these components to expose component-wise vulnerabilities while ensuring linguistic plausibility via perplexity-based filtering. Using this framework, four instruction-tuning datasets are structurally annotated and validated by human reviewers. Experiments across five advanced LLMs show that ComPerturb achieves state-of-the-art attack success rates, while ablation analyses confirm the complementary effects of prompt dissection and perplexity filtering. These results highlight the importance of structural awareness in evaluating and improving the adversarial robustness of LLMs.
VoxMind: An End-to-End Agentic Spoken Dialogue System
Tianle Liang | Yifu Chen | Shengpeng Ji | Yijun Chen | Zhiyang Jia | Jingyu Lu | Fan Zhuo | Xueyi Pu | Yangzhuo Li | Zhou Zhao
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Tianle Liang | Yifu Chen | Shengpeng Ji | Yijun Chen | Zhiyang Jia | Jingyu Lu | Fan Zhuo | Xueyi Pu | Yangzhuo Li | Zhou Zhao
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Recent end-to-end spoken dialogue models enable natural interaction. However, as user demands become increasingly complex, models that rely solely on conversational abilities often struggle to cope. Incorporating agentic capabilities is therefore essential: by enabling tool use, these models can extend their knowledge boundaries and better solve real-world tasks. Yet, existing research has largely concentrated on core perception and generation, with comparatively limited exploration of such tool-augmented extensions. To bridge this gap, we present VoxMind, an integrated framework designed to equip end-to-end spoken dialogue models with comprehensive agentic abilities. Leveraging our curated 470-hour AgentChat dataset, we incorporate a "Think-before-Speak" mechanism, enabling the model to internalize structured reasoning as a critical prerequisite for planning and response generation. Furthermore, to mitigate latency bottlenecks caused by large-scale tool integration, we propose a Multi-Agent Dynamic Tool Management architecture. By asynchronously delegating retrieval tasks to an auxiliary agent aligned with the main model’s reasoning trajectory, this system effectively decouples inference latency from toolset size. Experimental results confirm that VoxMind achieves significant improvements in agent performance: compared with strong baselines, the task completion rate increases from 34.88% to 74.57%, outperforming Gemini-2.5-Pro on spoken agent tasks while preserving general conversational quality. The source code and associated data are publicly available at https://github.com/MM-Speech/VoxMind.
SDiaReward: Modeling and Benchmarking Spoken Dialogue Rewards with Modality and Colloquialness
Jingyu Lu | Yuhan Wang | Fan Zhuo | Xize Cheng | Changhao Pan | Xueyi Pu | Yifu Chen | Chenyuhao Wen | Tianle Liang | Zhou Zhao
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Jingyu Lu | Yuhan Wang | Fan Zhuo | Xize Cheng | Changhao Pan | Xueyi Pu | Yifu Chen | Chenyuhao Wen | Tianle Liang | Zhou Zhao
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
The rapid evolution of end-to-end spoken dialogue systems demands transcending mere textual semantics to incorporate paralinguistic nuances and the spontaneous nature of human conversation. However, current methods struggle with two critical gaps: the modality gap, involving prosody and emotion, and the colloquialness gap, distinguishing written scripts from natural speech. To address these challenges, we introduce SDiaReward, an end-to-end multi-turn reward model trained on SDiaReward-Dataset, a novel collection of episode-level preference pairs explicitly targeting these gaps. It operates directly on full multi-turn speech episodes and is optimized with pairwise preference supervision, enabling joint assessment of modality and colloquialness in a single evaluator. We further establish ESDR-Bench, a stratified benchmark for robust episode-level evaluation. Experiments demonstrate that SDiaReward achieves state-of-the-art pairwise preference accuracy, significantly outperforming general-purpose audio LLMs. Further analysis suggests that SDiaReward captures relative conversational expressiveness beyond superficial synthesis cues, improving generalization across domains and recording conditions.
2025
Synthetic Singers: A Review of Deep-Learning-based Singing Voice Synthesis Approaches
Changhao Pan | Dongyu Yao | Yu Zhang | Wenxiang Guo | Jingyu Lu | Zhiyuan Zhu | Zhou Zhao
Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics
Changhao Pan | Dongyu Yao | Yu Zhang | Wenxiang Guo | Jingyu Lu | Zhiyuan Zhu | Zhou Zhao
Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics
Recent advances in singing voice synthesis (SVS) have attracted substantial attention from both academia and industry. With the advent of large language models and novel generative paradigms, producing controllable, high‐fidelity singing voices has become an attainable goal. Yet the field still lacks a comprehensive survey that systematically analyzes deep‐learning‐based singing voice systems and their enabling technologies.To address the aforementioned issue, this survey first categorizes existing systems by task type and then organizes current architectures into two major paradigms: cascaded and end-to-end approaches. Moreover, we provide an in-depth analysis of core technologies, covering singing modeling and control techniques. Finally, we review relevant datasets, annotation tools, and evaluation benchmarks that support training and assessment. In appendix, we introduce training strategies and further discussion of SVS. This survey provides an up-to-date review of the literature on SVS models, which would be a useful reference for both researchers and engineers. Related materials are available at https://github.com/David-Pigeon/SyntheticSingers.
Versatile Framework for Song Generation with Prompt-based Control
Yu Zhang | Wenxiang Guo | Changhao Pan | Zhiyuan Zhu | Ruiqi Li | Jingyu Lu | Rongjie Huang | Ruiyuan Zhang | Zhiqing Hong | Ziyue Jiang | Zhou Zhao
Findings of the Association for Computational Linguistics: EMNLP 2025
Yu Zhang | Wenxiang Guo | Changhao Pan | Zhiyuan Zhu | Ruiqi Li | Jingyu Lu | Rongjie Huang | Ruiyuan Zhang | Zhiqing Hong | Ziyue Jiang | Zhou Zhao
Findings of the Association for Computational Linguistics: EMNLP 2025
Song generation focuses on producing controllable high-quality songs based on various prompts. However, existing methods struggle to generate vocals and accompaniments with prompt-based control and proper alignment. Additionally, they fall short in supporting various tasks. To address these challenges, we introduce VersBand, a multi-task song generation framework for synthesizing high-quality, aligned songs with prompt-based control. VersBand comprises these primary models: 1) VocalBand, a decoupled model, leverages the flow-matching method for generating singing styles, pitches, and mel-spectrograms, allowing fast, high-quality vocal generation with style control. 2) AccompBand, a flow-based transformer model, incorporates the Band-MOE, selecting suitable experts for enhanced quality, alignment, and control. This model allows for generating controllable, high-quality accompaniments aligned with vocals. 3) Two generation models, LyricBand for lyrics and MelodyBand for melodies, contribute to the comprehensive multi-task song generation system, allowing for extensive control based on multiple prompts. Experimental results demonstrate that VersBand performs better over baseline models across multiple song generation tasks using objective and subjective metrics.
Search
Fix author
Co-authors
- Zhou Zhao 4
- Changhao Pan 3
- Yifu Chen 2
- Wenxiang Guo 2
- Tianle Liang 2
- Xueyi Pu 2
- Yu Zhang 2
- Zhiyuan Zhu 2
- Fan Zhuo 2
- Yijun Chen 1
- Xize Cheng 1
- Chuangxin Chu 1
- Yuyao Ge 1
- Zhiqing Hong 1
- Haotian Huang 1
- Rongjie Huang 1
- Yuekai Huang 1
- Shengpeng Ji 1
- Zhiyang Jia 1
- Ziyou Jiang 1
- Ziyue Jiang 1
- Mingyang Li 1
- Ruiqi Li 1
- Tianhao Li 1
- Yangzhuo Li 1
- Yuhan Wang 1
- Chenyuhao Wen 1
- Qian Xiong 1
- Dongyu Yao 1
- Tianyu Zeng 1
- Ruiyuan Zhang 1
- Yujia Zheng 1