Depth Aware Hierarchical Replay Continual Learning for Knowledge Based Question Answering
Zhixiong Cao, Hai-Tao Zheng, Yangning Li, Jin Xu, Rongsheng Li, Hong-Gee Kim
Abstract
Continual learning is an emerging area of machine learning that deals with the issue where models adapt well to the latest data but lose the ability to remember past data due to changes in the data source. A widely adopted solution is by keeping a small memory of previous learned data that use replay. Most of the previous studies on continual learning focused on classification tasks, such as image classification and text classification, where the model needs only to categorize the input data. Inspired by the human ability to incrementally learn knowledge and solve different problems using learned knowledge, we considered a more pratical scenario, knowledge based quesiton answering about continual learning. In this scenario, each single question is different from others(means different fact trippes to answer them) while classification tasks only need to find feature boundaries of different categories, which are the curves or surfaces that separate different categories in the feature space. To address this issue, we proposed a depth aware hierarchical replay framework which include a tree structure classfier to have a sense of knowledge distribution and fill the gap between text classfication tasks and question-answering tasks for continual learning, a local sampler to grasp these critical samples and a depth aware learning network to reconstructe the feature space of a single learning round. In our experiments, we have demonstrated that our proposed model outperforms previous continual learning methods in mitigating the issue of catastrophic forgetting.- Anthology ID:
- 2024.lrec-main.416
- Volume:
- Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)
- Month:
- May
- Year:
- 2024
- Address:
- Torino, Italia
- Editors:
- Nicoletta Calzolari, Min-Yen Kan, Veronique Hoste, Alessandro Lenci, Sakriani Sakti, Nianwen Xue
- Venues:
- LREC | COLING
- SIG:
- Publisher:
- ELRA and ICCL
- Note:
- Pages:
- 4654–4664
- Language:
- URL:
- https://aclanthology.org/2024.lrec-main.416
- DOI:
- Cite (ACL):
- Zhixiong Cao, Hai-Tao Zheng, Yangning Li, Jin Xu, Rongsheng Li, and Hong-Gee Kim. 2024. Depth Aware Hierarchical Replay Continual Learning for Knowledge Based Question Answering. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 4654–4664, Torino, Italia. ELRA and ICCL.
- Cite (Informal):
- Depth Aware Hierarchical Replay Continual Learning for Knowledge Based Question Answering (Cao et al., LREC-COLING 2024)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-3/2024.lrec-main.416.pdf