@inproceedings{zhu-etal-2024-ra2fd,
    title = "{RA}2{FD}: Distilling Faithfulness into Efficient Dialogue Systems",
    author = "Zhu, Zhiyuan  and
      Liao, Yusheng  and
      Xu, Chenxin  and
      Guan, Yunfeng  and
      Wang, Yanfeng  and
      Wang, Yu",
    editor = "Al-Onaizan, Yaser  and
      Bansal, Mohit  and
      Chen, Yun-Nung",
    booktitle = "Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2024",
    address = "Miami, Florida, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/jlcl-multiple-ingestion/2024.emnlp-main.685/",
    doi = "10.18653/v1/2024.emnlp-main.685",
    pages = "12304--12317",
    abstract = "Generating faithful and fast responses is crucial in the knowledge-grounded dialogue. Retrieval Augmented Generation (RAG) strategies are effective but are inference inefficient, while previous Retrieval Free Generations (RFG) are more efficient but sacrifice faithfulness. To solve this faithfulness-efficiency trade-off dilemma, we propose a novel retrieval-free model training scheme named Retrieval Augmented to Retrieval Free Distillation (RA2FD) to build a retrieval-free model that achieves higher faithfulness than the previous RFG method while maintaining inference efficiency. The core idea of RA2FD is to use a teacher-student framework to distill the faithfulness capacity of a teacher, which is an oracle RAG model that generates multiple knowledge-infused responses. The student retrieval-free model learns how to generate faithful responses from these teacher labels through sequence-level distillation and contrastive learning. Experiment results show that RA2FD let the faithfulness performance of an RFG model surpass the previous SOTA RFG baseline on three knowledge-grounded dialogue datasets by an average of 33{\%} and even matching an RAG model`s performance while significantly improving inference efficiency. Our code is available at https://github.com/zzysjtuiwct/RA2FD."
}