Jialin Li
2026
PrefIx: Understand and Adapt to User Preference in Human-Agent Interaction
Jialin Li | Zhenhao Chen | Hanjun Luo | Hanan Salam
Findings of the Association for Computational Linguistics: ACL 2026
Jialin Li | Zhenhao Chen | Hanjun Luo | Hanan Salam
Findings of the Association for Computational Linguistics: ACL 2026
LLM-based agents can complete tasks correctly yet still frustrate users through poor interaction patterns, such as excessive confirmations, opaque reasoning, or misaligned pacing. Current benchmarks evaluate task accuracy but overlook how agents interact: whether they infer preferences from implicit cues, adapt dynamically, or maintain fine-grained interaction quality. We introduce , a configurable environment that evaluates both what agents accomplish and how they interact. Central to is the Interaction-as-a-Tool (IaaT) paradigm, which treats interaction behaviors as structured tool calls, unifying them with existing evaluation frameworks. We define 31 preference settings across 14 attributes and formalize user experience (UX) as a core metric alongside task accuracy. A composite LLM-as-a-Judge mechanism across seven UX dimensions achieves strong aggregate reliability (ICC > 0.79), high internal consistency (𝛼 = 0.943), and human correlation (𝜌 = 0.52-0.78). Preference-aware agents show 7.6% average UX improvement and 18.5% gain in preference alignment.
2025
A Lightweight Multi Aspect Controlled Text Generation Solution For Large Language Models
Chenyang Zhang | Jiayi Lin | Haibo Tong | Bingxuan Hou | Dongyu Zhang | Jialin Li | Junli Wang
Proceedings of the 5th Workshop on Trustworthy NLP (TrustNLP 2025)
Chenyang Zhang | Jiayi Lin | Haibo Tong | Bingxuan Hou | Dongyu Zhang | Jialin Li | Junli Wang
Proceedings of the 5th Workshop on Trustworthy NLP (TrustNLP 2025)
Multi-Aspect Controllable Text Generation (MCTG) introduces fine-grained multiple constraints in natural language generation, i.e. control attributes in topics, sentiments, and detoxification.MCTG demonstrates application prospects for trustworthy generation of Large Language Models (LLMs) but is limited by generalization issues.Existing work exploits additional structures and strategies for solutions, requiring LLMs’ modifications.To activate LLMs’ MCTG ability, we propose a lightweight MCTG pipeline based on data augmentation and instruction tuning.We analyze aspect bias and correlations in traditional datasets and address these concerns with augmented control attributes and sentences.Augmented datasets are feasible for instruction tuning.We conduct experiments for various LLMs backbone and parameter sizes, demonstrating general effectiveness on MCTG performance.
Bold Claims or Self-Doubt? Factuality Hallucination Type Detection via Belief State
Dongyu Zhang | Qingqing Hong | Bingxuan Hou | Jiayi Lin | Chenyang Zhang | Jialin Li | Junli Wang
Findings of the Association for Computational Linguistics: EMNLP 2025
Dongyu Zhang | Qingqing Hong | Bingxuan Hou | Jiayi Lin | Chenyang Zhang | Jialin Li | Junli Wang
Findings of the Association for Computational Linguistics: EMNLP 2025
Large language models are prone to generating hallucination that deviates from factual information. Existing studies mainly focus on detecting the presence of hallucinations but lack a systematic classification approach, which hinders deeper exploration of their characteristics. To address this, we introduce the concept of belief state, which quantifies the model’s confidence in its own responses. We define the belief state of the model based on self-consistency, leveraging answer repetition rates to label confident and uncertain states. Based on this, we categorize factuality hallucination into two types: Overconfident Hallucination and Unaware Hallucination. Furthermore, we propose BAFH, a factuality hallucination type detection method. By training a classifier on model’s hidden states, we establish a link between hidden states and belief states, enabling efficient and automatic hallucination type detection. Experimental results demonstrate the effectiveness of BAFH and the differences between hallucination types.