A Large-Scale Corpus for Conversation Disentanglement

Jonathan K. Kummerfeld, Sai R. Gouravajhala, Joseph J. Peper, Vignesh Athreya, Chulaka Gunasekara, Jatin Ganhotra, Siva Sankalp Patel, Lazaros C Polymenakos, Walter Lasecki

Abstract
Disentangling conversations mixed together in a single stream of messages is a difficult task, made harder by the lack of large manually annotated datasets. We created a new dataset of 77,563 messages manually annotated with reply-structure graphs that both disentangle conversations and define internal conversation structure. Our data is 16 times larger than all previously released datasets combined, the first to include adjudication of annotation disagreements, and the first to include context. We use our data to re-examine prior work, in particular, finding that 89% of conversations in a widely used dialogue corpus are either missing messages or contain extra messages. Our manually-annotated data presents an opportunity to develop robust data-driven methods for conversation disentanglement, which will help advance dialogue research.
Anthology ID:
P19-1374
Volume:
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
Month:
July
Year:
2019
Address:
Florence, Italy
Editors:
Anna Korhonen, David Traum, Lluís Màrquez
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
3846–3856
Language:
URL:
https://aclanthology.org/P19-1374
DOI:
10.18653/v1/P19-1374
Bibkey:
Cite (ACL):
Jonathan K. Kummerfeld, Sai R. Gouravajhala, Joseph J. Peper, Vignesh Athreya, Chulaka Gunasekara, Jatin Ganhotra, Siva Sankalp Patel, Lazaros C Polymenakos, and Walter Lasecki. 2019. A Large-Scale Corpus for Conversation Disentanglement. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3846–3856, Florence, Italy. Association for Computational Linguistics.
Cite (Informal):
A Large-Scale Corpus for Conversation Disentanglement (Kummerfeld et al., ACL 2019)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-2024-clasp/P19-1374.pdf
Supplementary:
 P19-1374.Supplementary.pdf
Software:
 P19-1374.Software.tgz
Code
 jkkummerfeld/irc-disentanglement +  additional community code
Data
Ubuntu IRC, irc-disentanglement
PDF Search Code Supplementary Software