PrefIx: Understand and Adapt to User Preference in Human-Agent Interaction

Jialin Li, Zhenhao Chen, Hanjun Luo, Hanan Salam


Abstract
LLM-based agents can complete tasks correctly yet still frustrate users through poor interaction patterns, such as excessive confirmations, opaque reasoning, or misaligned pacing. Current benchmarks evaluate task accuracy but overlook how agents interact: whether they infer preferences from implicit cues, adapt dynamically, or maintain fine-grained interaction quality. We introduce , a configurable environment that evaluates both what agents accomplish and how they interact. Central to  is the Interaction-as-a-Tool (IaaT) paradigm, which treats interaction behaviors as structured tool calls, unifying them with existing evaluation frameworks. We define 31 preference settings across 14 attributes and formalize user experience (UX) as a core metric alongside task accuracy. A composite LLM-as-a-Judge mechanism across seven UX dimensions achieves strong aggregate reliability (ICC > 0.79), high internal consistency (𝛼 = 0.943), and human correlation (𝜌 = 0.52-0.78). Preference-aware agents show 7.6% average UX improvement and 18.5% gain in preference alignment.
Anthology ID:
2026.findings-acl.1506
Volume:
Findings of the Association for Computational Linguistics: ACL 2026
Month:
July
Year:
2026
Address:
San Diego, California, United States
Editors:
Maria Liakata, Viviane P. Moreira, Jiajun Zhang, David Jurgens
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
30110–30149
Language:
URL:
https://preview.aclanthology.org/ingest-acl/2026.findings-acl.1506/
DOI:
Bibkey:
Cite (ACL):
Jialin Li, Zhenhao Chen, Hanjun Luo, and Hanan Salam. 2026. PrefIx: Understand and Adapt to User Preference in Human-Agent Interaction. In Findings of the Association for Computational Linguistics: ACL 2026, pages 30110–30149, San Diego, California, United States. Association for Computational Linguistics.
Cite (Informal):
PrefIx: Understand and Adapt to User Preference in Human-Agent Interaction (Li et al., Findings 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl/2026.findings-acl.1506.pdf
Checklist:
 2026.findings-acl.1506.checklist.pdf