Abstract
Paraphrase generation aims to improve the clarity of a sentence by using different wording that convey similar meaning. For better quality of generated paraphrases, we propose a framework that combines the effectiveness of two models – transformer and sequence-to-sequence (seq2seq). We design a two-layer stack of encoders. The first layer is a transformer model containing 6 stacked identical layers with multi-head self attention, while the second-layer is a seq2seq model with gated recurrent units (GRU-RNN). The transformer encoder layer learns to capture long-term dependencies, together with syntactic and semantic properties of the input sentence. This rich vector representation learned by the transformer serves as input to the GRU-RNN encoder responsible for producing the state vector for decoding. Experimental results on two datasets-QUORA and MSCOCO using our framework, produces a new benchmark for paraphrase generation.- Anthology ID:
- D19-5627
- Volume:
- Proceedings of the 3rd Workshop on Neural Generation and Translation
- Month:
- November
- Year:
- 2019
- Address:
- Hong Kong
- Editors:
- Alexandra Birch, Andrew Finch, Hiroaki Hayashi, Ioannis Konstas, Thang Luong, Graham Neubig, Yusuke Oda, Katsuhito Sudoh
- Venue:
- NGT
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 249–255
- Language:
- URL:
- https://aclanthology.org/D19-5627
- DOI:
- 10.18653/v1/D19-5627
- Cite (ACL):
- Elozino Egonmwan and Yllias Chali. 2019. Transformer and seq2seq model for Paraphrase Generation. In Proceedings of the 3rd Workshop on Neural Generation and Translation, pages 249–255, Hong Kong. Association for Computational Linguistics.
- Cite (Informal):
- Transformer and seq2seq model for Paraphrase Generation (Egonmwan & Chali, NGT 2019)
- PDF:
- https://preview.aclanthology.org/ingest-acl-2023-videos/D19-5627.pdf