@inproceedings{liu-etal-2025-tell,
    title = "Tell Me What You Don{'}t Know: Enhancing Refusal Capabilities of Role-Playing Agents via Representation Space Analysis and Editing",
    author = "Liu, Wenhao  and
      An, Siyu  and
      Lu, Junru  and
      Wu, Muling  and
      Li, Tianlong  and
      Wang, Xiaohua  and
      Lv, Changze  and
      Zheng, Xiaoqing  and
      Yin, Di  and
      Sun, Xing  and
      Huang, Xuanjing",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Findings of the Association for Computational Linguistics: ACL 2025",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/display_plenaries/2025.findings-acl.311/",
    pages = "5983--6005",
    ISBN = "979-8-89176-256-5",
    abstract = "Role-Playing Agents (RPAs) have shown remarkable performance in various applications, yet they often struggle to recognize and appropriately respond to hard queries that conflict with their role-play knowledge. To investigate RPAs' performance when faced with different types of conflicting requests, we develop an evaluation benchmark that includes contextual knowledge conflicting requests, parametric knowledge conflicting requests, and non-conflicting requests to assess RPAs' ability to identify conflicts and refuse to answer appropriately without over-refusing. Through extensive evaluation, we find that most RPAs behave significant performance gaps toward different conflict requests. To elucidate the reasons, we conduct an in-depth representation-level analysis of RPAs under various conflict scenarios. Our findings reveal the existence of rejection regions and direct response regions within the model{'}s forwarding representation, and thus influence the RPA{'}s final response behavior. Therefore, we introduce a lightweight representation editing approach that conveniently shifts conflicting requests to the rejection region, thereby enhancing the model{'}s refusal accuracy. The extensive experiments validate the effectiveness of our editing method, improving RPAs' refusal ability of conflicting requests while maintaining their general role-playing capabilities."
}