Document Modeling with External Attention for Sentence Extraction
Shashi Narayan, Ronald Cardenas, Nikos Papasarantopoulos, Shay B. Cohen, Mirella Lapata, Jiangsheng Yu, Yi Chang
Abstract
Document modeling is essential to a variety of natural language understanding tasks. We propose to use external information to improve document modeling for problems that can be framed as sentence extraction. We develop a framework composed of a hierarchical document encoder and an attention-based extractor with attention over external information. We evaluate our model on extractive document summarization (where the external information is image captions and the title of the document) and answer selection (where the external information is a question). We show that our model consistently outperforms strong baselines, in terms of both informativeness and fluency (for CNN document summarization) and achieves state-of-the-art results for answer selection on WikiQA and NewsQA.- Anthology ID:
- P18-1188
- Erratum e1:
- P18-1188e1
- Volume:
- Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
- Month:
- July
- Year:
- 2018
- Address:
- Melbourne, Australia
- Editors:
- Iryna Gurevych, Yusuke Miyao
- Venue:
- ACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 2020–2030
- Language:
- URL:
- https://aclanthology.org/P18-1188
- DOI:
- 10.18653/v1/P18-1188
- Cite (ACL):
- Shashi Narayan, Ronald Cardenas, Nikos Papasarantopoulos, Shay B. Cohen, Mirella Lapata, Jiangsheng Yu, and Yi Chang. 2018. Document Modeling with External Attention for Sentence Extraction. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2020–2030, Melbourne, Australia. Association for Computational Linguistics.
- Cite (Informal):
- Document Modeling with External Attention for Sentence Extraction (Narayan et al., ACL 2018)
- PDF:
- https://preview.aclanthology.org/emnlp22-frontmatter/P18-1188.pdf
- Code
- shashiongithub/Document-Models-with-Ext-Information
- Data
- MS MARCO, NewsQA, SQuAD, WikiQA