Abstract
Paraphrase generation is an interesting and challenging NLP task which has numerous practical applications. In this paper, we analyze datasets commonly used for paraphrase generation research, and show that simply parroting input sentences surpasses state-of-the-art models in the literature when evaluated on standard metrics. Our findings illustrate that a model could be seemingly adept at generating paraphrases, despite only making trivial changes to the input sentence or even none at all.- Anthology ID:
- D19-1611
- Volume:
- Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
- Month:
- November
- Year:
- 2019
- Address:
- Hong Kong, China
- Editors:
- Kentaro Inui, Jing Jiang, Vincent Ng, Xiaojun Wan
- Venues:
- EMNLP | IJCNLP
- SIG:
- SIGDAT
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 5960–5968
- Language:
- URL:
- https://aclanthology.org/D19-1611
- DOI:
- 10.18653/v1/D19-1611
- Cite (ACL):
- Hong-Ren Mao and Hung-Yi Lee. 2019. Polly Want a Cracker: Analyzing Performance of Parroting on Paraphrase Generation Datasets. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 5960–5968, Hong Kong, China. Association for Computational Linguistics.
- Cite (Informal):
- Polly Want a Cracker: Analyzing Performance of Parroting on Paraphrase Generation Datasets (Mao & Lee, EMNLP-IJCNLP 2019)
- PDF:
- https://preview.aclanthology.org/naacl24-info/D19-1611.pdf
- Data
- MS COCO