@inproceedings{sun-etal-2025-divide,
    title = "Divide-Then-Align: Honest Alignment based on the Knowledge Boundary of {RAG}",
    author = "Sun, Xin  and
      Xie, Jianan  and
      Chen, Zhongqi  and
      Liu, Qiang  and
      Wu, Shu  and
      Chen, Yuehe  and
      Song, Bowen  and
      Wang, Zilei  and
      Wang, Weiqiang  and
      Wang, Liang",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingestion-acl-25/2025.acl-long.561/",
    pages = "11461--11480",
    ISBN = "979-8-89176-251-0",
    abstract = "Large language models (LLMs) augmented with retrieval systems have significantly advanced natural language processing tasks by integrating external knowledge sources, enabling more accurate and contextually rich responses. To improve the robustness of such systems against noisy retrievals, Retrieval-Augmented Fine-Tuning (RAFT) has emerged as a widely adopted method. However, RAFT conditions models to generate answers even in the absence of reliable knowledge. This behavior undermines their reliability in high-stakes domains, where acknowledging uncertainty is critical. To address this issue, we propose Divide-Then-Align (DTA), a post-training approach designed to endow RAG systems with the ability to respond with ``I don{'}t know'' when the query is out of the knowledge boundary of both the retrieved passages and the model{'}s internal knowledge. DTA divides data samples into four knowledge quadrants and constructs tailored preference data for each quadrant, resulting in a curated dataset for Direct Preference Optimization (DPO). Experimental results on three benchmark datasets demonstrate that effectively balances accuracy with appropriate abstention, enhancing the reliability and trustworthiness of retrieval-augmented systems."
}