@inproceedings{tamoyan-etal-2025-llm,
    title = "{LLM} Roleplay: Simulating Human-Chatbot Interaction",
    author = "Tamoyan, Hovhannes  and
      Schuff, Hendrik  and
      Gurevych, Iryna",
    editor = "Hale, James  and
      Deuksin Kwon, Brian  and
      Dutt, Ritam",
    booktitle = "Proceedings of the Third Workshop on Social Influence in Conversations (SICon 2025)",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/landing_page/2025.sicon-1.1/",
    pages = "1--26",
    ISBN = "979-8-89176-266-4",
    abstract = "The development of chatbots requires collecting a large number of human-chatbot dialogues to reflect the breadth of users' sociodemographic backgrounds and conversational goals. However, the resource requirements to conduct the respective user studies can be prohibitively high and often only allow for a narrow analysis of specific dialogue goals and participant demographics. In this paper, we propose LLM Roleplay, the first comprehensive method integrating multi-turn human-chatbot interaction simulation, explicit persona construction from sociodemographic traits, goal-driven dialogue planning, and robust handling of conversational failures, enabling broad utility and reliable dialogue generation. To validate our method, we collect natural human-chatbot dialogues from different sociodemographic groups and conduct a user study to compare these with our generated dialogues. We evaluate the capabilities of state-of-the-art LLMs in maintaining a conversation during their embodiment of a specific persona and find that our method can simulate human-chatbot dialogues with a high indistinguishability rate."
}