@inproceedings{zhan-etal-2023-user,
    title = "User Simulator Assisted Open-ended Conversational Recommendation System",
    author = "Zhan, Qiusi  and
      Guo, Xiaojie  and
      Ji, Heng  and
      Wu, Lingfei",
    editor = "Chen, Yun-Nung  and
      Rastogi, Abhinav",
    booktitle = "Proceedings of the 5th Workshop on NLP for Conversational AI (NLP4ConvAI 2023)",
    month = jul,
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/jlcl-multiple-ingestion/2023.nlp4convai-1.8/",
    doi = "10.18653/v1/2023.nlp4convai-1.8",
    pages = "89--101",
    abstract = "Conversational recommendation systems (CRS) have gained popularity in e-commerce as they can recommend items during user interactions. However, current open-ended CRS have limited recommendation performance due to their short-sighted training process, which only predicts one utterance at a time without considering its future impact. To address this, we propose a User Simulator (US) that communicates with the CRS using natural language based on given user preferences, enabling long-term reinforcement learning. We also introduce a framework that uses reinforcement learning (RL) with two novel rewards, i.e., recommendation and conversation rewards, to train the CRS. This approach considers the long-term goals and improves both the conversation and recommendation performance of the CRS. Our experiments show that our proposed framework improves the recall of recommendations by almost 100{\%}. Moreover, human evaluation demonstrates the superiority of our framework in enhancing the informativeness of generated utterances."
}