Leveraging Large Language Models for Conversational Multi-Doc Question Answering: The First Place of WSDM Cup 2024

Yiming Li, Zhao Zhang


Abstract
Conversational multi-doc question answering aims to answer specific questions based on the retrieved documents as well as the contextual conversations. In this paper, we introduce our winning approach for the “Conversational Multi-Doc QA” challenge in WSDM Cup 2024, which exploits the superior natural language understanding and generation capability of Large Language Models (LLMs). We first adapt LLMs to the task, then devise a hybrid training strategy to make the most of in-domain unlabeled data. Moreover, an advanced text embedding model is adopted to filter out potentially irrelevant documents, and several approaches are designed and compared for the model ensemble. Equipped with all these techniques, our solution finally ranked 1st place in WSDM Cup 2024, surpassing its rivals to a large extent. The source codes have been released at https://github.com/zhangzhao219/WSDM-Cup-2024.
Anthology ID:
2025.findings-acl.19
Volume:
Findings of the Association for Computational Linguistics: ACL 2025
Month:
July
Year:
2025
Address:
Vienna, Austria
Editors:
Wanxiang Che, Joyce Nabende, Ekaterina Shutova, Mohammad Taher Pilehvar
Venues:
Findings | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
349–355
Language:
URL:
https://preview.aclanthology.org/ingestion-acl-25/2025.findings-acl.19/
DOI:
Bibkey:
Cite (ACL):
Yiming Li and Zhao Zhang. 2025. Leveraging Large Language Models for Conversational Multi-Doc Question Answering: The First Place of WSDM Cup 2024. In Findings of the Association for Computational Linguistics: ACL 2025, pages 349–355, Vienna, Austria. Association for Computational Linguistics.
Cite (Informal):
Leveraging Large Language Models for Conversational Multi-Doc Question Answering: The First Place of WSDM Cup 2024 (Li & Zhang, Findings 2025)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingestion-acl-25/2025.findings-acl.19.pdf