Generating and Evaluating Summaries for Partial Email Threads: Conversational Bayesian Surprise and Silver Standards
Jordon Johnson, Vaden Masrani, Giuseppe Carenini, Raymond Ng
Abstract
We define and motivate the problem of summarizing partial email threads. This problem introduces the challenge of generating reference summaries for partial threads when human annotation is only available for the threads as a whole, particularly when the human-selected sentences are not uniformly distributed within the threads. We propose an oracular algorithm for generating these reference summaries with arbitrary length, and we are making the resulting dataset publicly available. In addition, we apply a recent unsupervised method based on Bayesian Surprise that incorporates background knowledge into partial thread summarization, extend it with conversational features, and modify the mechanism by which it handles redundancy. Experiments with our method indicate improved performance over the baseline for shorter partial threads; and our results suggest that the potential benefits of background knowledge to partial thread summarization should be further investigated with larger datasets.- Anthology ID:
- W17-5532
- Volume:
- Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue
- Month:
- August
- Year:
- 2017
- Address:
- Saarbrücken, Germany
- Editors:
- Kristiina Jokinen, Manfred Stede, David DeVault, Annie Louis
- Venue:
- SIGDIAL
- SIG:
- SIGDIAL
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 263–272
- Language:
- URL:
- https://preview.aclanthology.org/build-pipeline-with-new-library/W17-5532/
- DOI:
- 10.18653/v1/W17-5532
- Cite (ACL):
- Jordon Johnson, Vaden Masrani, Giuseppe Carenini, and Raymond Ng. 2017. Generating and Evaluating Summaries for Partial Email Threads: Conversational Bayesian Surprise and Silver Standards. In Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue, pages 263–272, Saarbrücken, Germany. Association for Computational Linguistics.
- Cite (Informal):
- Generating and Evaluating Summaries for Partial Email Threads: Conversational Bayesian Surprise and Silver Standards (Johnson et al., SIGDIAL 2017)
- PDF:
- https://preview.aclanthology.org/build-pipeline-with-new-library/W17-5532.pdf