Original Semantics-Oriented Attention and Deep Fusion Network for Sentence Matching

Mingtong Liu, Yujie Zhang, Jinan Xu, Yufeng Chen

Abstract
Sentence matching is a key issue in natural language inference and paraphrase identification. Despite the recent progress on multi-layered neural network with cross sentence attention, one sentence learns attention to the intermediate representations of another sentence, which are propagated from preceding layers and therefore are uncertain and unstable for matching, particularly at the risk of error propagation. In this paper, we present an original semantics-oriented attention and deep fusion network (OSOA-DFN) for sentence matching. Unlike existing models, each attention layer of OSOA-DFN is oriented to the original semantic representation of another sentence, which captures the relevant information from a fixed matching target. The multiple attention layers allow one sentence to repeatedly read the important information of another sentence for better matching. We then additionally design deep fusion to propagate the attention information at each matching layer. At last, we introduce a self-attention mechanism to capture global context to enhance attention-aware representation within each sentence. Experiment results on three sentence matching benchmark datasets SNLI, SciTail and Quora show that OSOA-DFN has the ability to model sentence matching more precisely.
Anthology ID:
D19-1267
Volume:
Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
Month:
November
Year:
2019
Address:
Hong Kong, China
Editors:
Kentaro Inui, Jing Jiang, Vincent Ng, Xiaojun Wan
Venues:
EMNLP | IJCNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
2652–2661
Language:
URL:
https://aclanthology.org/D19-1267
DOI:
10.18653/v1/D19-1267
Bibkey:
Cite (ACL):
Mingtong Liu, Yujie Zhang, Jinan Xu, and Yufeng Chen. 2019. Original Semantics-Oriented Attention and Deep Fusion Network for Sentence Matching. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2652–2661, Hong Kong, China. Association for Computational Linguistics.
Cite (Informal):
Original Semantics-Oriented Attention and Deep Fusion Network for Sentence Matching (Liu et al., EMNLP-IJCNLP 2019)
Copy Citation:
PDF:
https://preview.aclanthology.org/naacl24-info/D19-1267.pdf
PDF Search