@inproceedings{cai-etal-2026-ievoagent,
    title = "{IE}vo{A}gent: Evolving Conversational Agent based on User Implicit Feedback",
    author = "Cai, Yichen  and
      Li, Jiayang  and
      Qiu, Junyuan  and
      Guo, Jingya  and
      You, Weitao  and
      Yang, Changyuan  and
      Sun, Lingyun  and
      Chen, Pei",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-long.441/",
    pages = "9725--9743",
    ISBN = "979-8-89176-390-6",
    abstract = "Current conversational agents often follow static learning paradigms and miss the implicit, evolving feedback embedded in users' follow-up behaviors. We propose IEvoAgent, an evolving conversational agent framework that leverages the structured dependency between agent responses and user reactions. We construct an annotated dataset from LMSYS-Chat-1M and WildChat and find consistent response-conditioned feedback patterns. Based on this finding, IEvoAgent uses a conditional feedback distribution matrix to estimate expected feedback rewards, combining offline KTO alignment with an inference-time prompt-evolution mechanism driven by a dynamic matrix. Experiments on MT-Bench-101, WildBench, and FB-Bench show improvements over open-source baselines, indicating that mining implicit feedback supports better multi-turn alignment under evolving user preferences. Our code and dataset are available at https://github.com/Hualeez/IEvoAgent."
}