@inproceedings{yang-etal-2024-chamain,
    title = "Chamain: Harmonizing Character Persona Integrity with Domain-Adaptive Knowledge in Dialogue Generation",
    author = "Yang, Seung-Moo  and
      Lee, Jeehyun  and
      Cho, Won Ik",
    editor = "Nouri, Elnaz  and
      Rastogi, Abhinav  and
      Spithourakis, Georgios  and
      Liu, Bing  and
      Chen, Yun-Nung  and
      Li, Yu  and
      Albalak, Alon  and
      Wakaki, Hiromi  and
      Papangelis, Alexandros",
    booktitle = "Proceedings of the 6th Workshop on NLP for Conversational AI (NLP4ConvAI 2024)",
    month = aug,
    year = "2024",
    address = "Bangkok, Thailand",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/jlcl-multiple-ingestion/2024.nlp4convai-1.7/",
    pages = "101--113",
    abstract = "Recent advances in large language models (LLMs) have shown their capacity for generating natural dialogues, leveraging extensive pre-trained knowledge. However, the seamless integration of domain-specific knowledge into dialogue agents, without undermining their personas or unique textual style, remains a challenging task. Traditional approaches, such as constructing knowledge-aware character dialogue datasets or training LLMs from the ground up, require considerable resources. Sequentially fine-tuning character chatbots across multiple datasets or applying existing merging techniques often leads to catastrophic forgetting, resulting in the loss of both knowledge and the character`s distinct persona. This compromises the model`s ability to consistently generate character-driven dialogues within a user-centric framework. In this context, we introduce a novel model merging method, Chamain, which effortlessly enhances the performance of character models, much like finding a {\textquotedblleft}free lunch{\textquotedblright}. Chamain merges domain-specific knowledge into a character model by parameter-wise weight combination of instruction-tuned models and learns to reflect persona`s unique characteristics and style through Layer-wise merging. Our experiments demonstrate that Chamain effectively maintains style while also solving domain-specific problems to a certain extent compared to the baselines, even showing a higher style probability compared to the character model in legal QA."
}