@inproceedings{fu-etal-2025-dacp,
    title = "{DACP}: Domain-Adaptive Continual Pre-Training of Large Language Models for Phone Conversation Summarization",
    author = "Fu, Xue-Yong  and
      Khasanova, Elena  and
      Laskar, Md Tahmid Rahman  and
      Saini, Harsh  and
      Tn, Shashi Bhushan",
    editor = "Dong, Yue  and
      Xiao, Wen  and
      Zhang, Haopeng  and
      Zhang, Rui  and
      Ernst, Ori  and
      Wang, Lu  and
      Liu, Fei",
    booktitle = "Proceedings of The 5th New Frontiers in Summarization Workshop",
    month = nov,
    year = "2025",
    address = "Hybrid",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2025.newsum-main.7/",
    pages = "94--101",
    ISBN = "979-8-89176-337-1",
    abstract = "Large language models (LLMs) have achieved impressive performance in text summarization, yet their performance often falls short when applied to specialized domains that differ from their original pre-training distribution. While fine-tuning can improve summarization quality, it typically relies on costly and scarce high-quality labeled data. In this work, we explore continual pre-training as a scalable, self-supervised approach to adapt LLMs for downstream summarization tasks, particularly in the context of noisy real-world conversation transcripts. We conduct extensive experiments using large-scale, unlabeled business conversation data to investigate whether continual pre-training enhances model capabilities in conversational summarization. Our results demonstrate that continual pre-training yields substantial gains in both in-domain and out-of-domain summarization benchmarks, while maintaining strong generalization and robustness. We also analyze the effects of data selection strategies, providing practical guidelines for applying continual pre-training in summarization-focused industrial applications."
}