@inproceedings{balepur-etal-2025-mods,
    title = "{M}o{DS}: Moderating a Mixture of Document Speakers to Summarize Debatable Queries in Document Collections",
    author = "Balepur, Nishant  and
      Siu, Alexa  and
      Lipka, Nedim  and
      Dernoncourt, Franck  and
      Sun, Tong  and
      Boyd-Graber, Jordan Lee  and
      Mathur, Puneet",
    editor = "Chiruzzo, Luis  and
      Ritter, Alan  and
      Wang, Lu",
    booktitle = "Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)",
    month = apr,
    year = "2025",
    address = "Albuquerque, New Mexico",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/landing_page/2025.naacl-long.20/",
    pages = "465--491",
    ISBN = "979-8-89176-189-6",
    abstract = "Query-focused summarization (QFS) gives a summary of documents to answer a query.Past QFS work assumes queries have one answer, ignoring debatable ones (*Is law school worth it?*).We introduce **Debatable QFS (DQFS)**, a task to create summaries that answer debatable queries via documents with opposing perspectives; summaries must *comprehensively cover* all sources and *balance perspectives*, favoring no side.These goals elude LLM QFS systems, which: 1) lack structured content plans, failing to guide LLMs to write balanced summaries, and 2) employ the same query to retrieve contexts across documents, failing to cover all perspectives specific to each document{'}s content.To overcome this, we design MoDS, a multi-LLM framework mirroring human panel discussions.MoDS treats documents as individual Speaker LLMs and has a Moderator LLM that picks speakers to respond to tailored queries for planned topics.Speakers use tailored queries to retrieve relevant contexts from their documents and supply perspectives, which are tracked in a rich outline, yielding a content plan to guide the final summary.Experiments on ConflictingQA with controversial web queries and DebateQFS, our new dataset of debate queries from Debatepedia, show MoDS beats SOTA by 38-59{\%} in topic paragraph coverage and balance, based on new citation metrics. Users also find MoDS{'}s summaries to be readable and more balanced."
}