Non-compositional Expression Generation Based on Curriculum Learning and Continual Learning

Jianing Zhou, Ziheng Zeng, Hongyu Gong, Suma Bhat

Abstract
Non-compositional expressions, by virtue of their non-compositionality, are a classic ‘pain in the neck’ for NLP systems. Different from the general language modeling and generation tasks that are primarily compositional, generating non-compositional expressions is more challenging for current neural models, including large pre-trained language models. The main reasons are 1) their non-compositionality, and 2) the limited data resources. Therefore, to make the best use of available data for modeling non-compositionality, we propose a dynamic curriculum learning framework, which learns training examples from easy ones to harder ones thus optimizing the learning step by step but suffers from the forgetting problem. To alleviate the forgetting problem brought by the arrangement of training examples, we also apply a continual learning method into our curriculum learning framework. Our proposed method combined curriculum and continual learning, to gradually improve the model’s performance on the task of non-compositional expression generation. Experiments on idiomatic expression generation and metaphor generation affirm the effectiveness of our proposed curriculum learning framework and the application of continual learning. Our codes are available at https://github.com/zhjjn/CL2Gen.git.
Anthology ID:
2023.findings-emnlp.286
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2023
Month:
December
Year:
2023
Address:
Singapore
Editors:
Houda Bouamor, Juan Pino, Kalika Bali
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
4320–4335
Language:
URL:
https://aclanthology.org/2023.findings-emnlp.286
DOI:
10.18653/v1/2023.findings-emnlp.286
Bibkey:
Cite (ACL):
Jianing Zhou, Ziheng Zeng, Hongyu Gong, and Suma Bhat. 2023. Non-compositional Expression Generation Based on Curriculum Learning and Continual Learning. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 4320–4335, Singapore. Association for Computational Linguistics.
Cite (Informal):
Non-compositional Expression Generation Based on Curriculum Learning and Continual Learning (Zhou et al., Findings 2023)
Copy Citation:
PDF:
https://preview.aclanthology.org/nschneid-patch-1/2023.findings-emnlp.286.pdf
PDF Search