Summary-Source Proposition-level Alignment: Task, Datasets and Supervised Baseline
Ori Ernst, Ori Shapira, Ramakanth Pasunuru, Michael Lepioshkin, Jacob Goldberger, Mohit Bansal, Ido Dagan
Abstract
Aligning sentences in a reference summary with their counterparts in source documents was shown as a useful auxiliary summarization task, notably for generating training data for salience detection. Despite its assessed utility, the alignment step was mostly approached with heuristic unsupervised methods, typically ROUGE-based, and was never independently optimized or evaluated. In this paper, we propose establishing summary-source alignment as an explicit task, while introducing two major novelties: (1) applying it at the more accurate proposition span level, and (2) approaching it as a supervised classification task. To that end, we created a novel training dataset for proposition-level alignment, derived automatically from available summarization evaluation data. In addition, we crowdsourced dev and test datasets, enabling model development and proper evaluation. Utilizing these data, we present a supervised proposition alignment baseline model, showing improved alignment-quality over the unsupervised approach.- Anthology ID:
- 2021.conll-1.25
- Volume:
- Proceedings of the 25th Conference on Computational Natural Language Learning
- Month:
- November
- Year:
- 2021
- Address:
- Online
- Venue:
- CoNLL
- SIG:
- SIGNLL
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 310–322
- Language:
- URL:
- https://aclanthology.org/2021.conll-1.25
- DOI:
- 10.18653/v1/2021.conll-1.25
- Cite (ACL):
- Ori Ernst, Ori Shapira, Ramakanth Pasunuru, Michael Lepioshkin, Jacob Goldberger, Mohit Bansal, and Ido Dagan. 2021. Summary-Source Proposition-level Alignment: Task, Datasets and Supervised Baseline. In Proceedings of the 25th Conference on Computational Natural Language Learning, pages 310–322, Online. Association for Computational Linguistics.
- Cite (Informal):
- Summary-Source Proposition-level Alignment: Task, Datasets and Supervised Baseline (Ernst et al., CoNLL 2021)
- PDF:
- https://preview.aclanthology.org/ingestion-script-update/2021.conll-1.25.pdf
- Code
- oriern/SuperPAL
- Data
- Multi-News