@inproceedings{zhao-etal-2025-dream,
    title = "Dream to Chat: Model-based Reinforcement Learning on Dialogues with User Belief Modeling",
    author = "Zhao, Yue  and
      Wang, Xiaoyu  and
      Wang, Dan  and
      Jiang, Zhonglin  and
      Gu, Qingqing  and
      Chen, Teng  and
      Xi, Ningyuan  and
      Qu, Jinxian  and
      Chen, Yong  and
      Ji, Luo",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2025",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/name-variant-enfa-fane/2025.findings-emnlp.256/",
    doi = "10.18653/v1/2025.findings-emnlp.256",
    pages = "4764--4781",
    ISBN = "979-8-89176-335-7",
    abstract = "World models have been widely utilized in robotics, gaming, and autonomous driving. However, their applications to natural language tasks are relatively limited. In this paper, we construct the dialogue world model, which could predict future utterances and user beliefs, including emotion, sentiment, and intention. In this paper, we propose a framework called DreamCUB, which shows that this user belief modeling and the entire dialogue world model can be established by LLM post-training. By defining a POMDP, we apply model-based reinforcement learning to the dialogue system and solve it by maximizing the information bottleneck. Experiments show that the pretrained dialogue world model can achieve state-of-the-art performances on emotion classification and sentiment identification, while dialogue quality is also enhanced by joint training of policy, critic and dialogue world model. Further analysis reveals that DreamCUB holds a reasonable exploration-exploitation balance and also transfers well to out-of-domain scenarios such as empathetic dialogues."
}