Generative Replay Inspired by Hippocampal Memory Indexing for Continual Language Learning
Aru Maekawa, Hidetaka Kamigaito, Kotaro Funakoshi, Manabu Okumura
Abstract
Continual learning aims to accumulate knowledge to solve new tasks without catastrophic forgetting for previously learned tasks. Research on continual learning has led to the development of generative replay, which prevents catastrophic forgetting by generating pseudo-samples for previous tasks and learning them together with new tasks. Inspired by the biological brain, we propose the hippocampal memory indexing to enhance the generative replay by controlling sample generation using compressed features of previous training samples. It enables the generation of a specific training sample from previous tasks, thus improving the balance and quality of generated replay samples. Experimental results indicate that our method effectively controls the sample generation and consistently outperforms the performance of current generative replay methods.- Anthology ID:
- 2023.eacl-main.65
- Volume:
- Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics
- Month:
- May
- Year:
- 2023
- Address:
- Dubrovnik, Croatia
- Venue:
- EACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 930–942
- Language:
- URL:
- https://aclanthology.org/2023.eacl-main.65
- DOI:
- Cite (ACL):
- Aru Maekawa, Hidetaka Kamigaito, Kotaro Funakoshi, and Manabu Okumura. 2023. Generative Replay Inspired by Hippocampal Memory Indexing for Continual Language Learning. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 930–942, Dubrovnik, Croatia. Association for Computational Linguistics.
- Cite (Informal):
- Generative Replay Inspired by Hippocampal Memory Indexing for Continual Language Learning (Maekawa et al., EACL 2023)
- PDF:
- https://preview.aclanthology.org/paclic-22-ingestion/2023.eacl-main.65.pdf