@inproceedings{nguyen-etal-2022-make,
    title = "Make The Most of Prior Data: A Solution for Interactive Text Summarization with Preference Feedback",
    author = "Nguyen, Duy-Hung  and
      Nghiem, Nguyen Viet Dung  and
      Nguyen, Bao-Sinh  and
      Tien Le, Dung Tien  and
      Sabahi, Shahab  and
      Nguyen, Minh-Tien  and
      Le, Hung",
    editor = "Carpuat, Marine  and
      de Marneffe, Marie-Catherine  and
      Meza Ruiz, Ivan Vladimir",
    booktitle = "Findings of the Association for Computational Linguistics: NAACL 2022",
    month = jul,
    year = "2022",
    address = "Seattle, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/jlcl-multiple-ingestion/2022.findings-naacl.147/",
    doi = "10.18653/v1/2022.findings-naacl.147",
    pages = "1919--1930",
    abstract = "For summarization, human preferences is critical to tame outputs of the summarizer in favor of human interests, as ground-truth summaries are scarce and ambiguous. Practical settings require dynamic exchanges between humans and AI agents wherein feedback is provided in an online manner, a few at a time. In this paper, we introduce a new framework to train summarization models with preference feedback interactively. By properly leveraging offline data and a novel reward model, we improve the performance regarding ROUGE scores and sample-efficiency. Our experiments on three various datasets confirm the benefit of the proposed framework in active, few-shot and online settings of preference learning."
}