Abstract
This paper presents a way to inject and leverage existing knowledge from external sources in a Deep Learning environment, extending the recently proposed Recurrent Independent Mechnisms (RIMs) architecture, which comprises a set of interacting yet independent modules. We show that this extension of the RIMs architecture is an effective framework with lower parameter implications compared to purely fine-tuned systems.- Anthology ID:
- 2021.deelio-1.11
- Volume:
- Proceedings of Deep Learning Inside Out (DeeLIO): The 2nd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures
- Month:
- June
- Year:
- 2021
- Address:
- Online
- Editors:
- Eneko Agirre, Marianna Apidianaki, Ivan Vulić
- Venue:
- DeeLIO
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 108–118
- Language:
- URL:
- https://aclanthology.org/2021.deelio-1.11
- DOI:
- 10.18653/v1/2021.deelio-1.11
- Cite (ACL):
- Parsa Bagherzadeh and Sabine Bergler. 2021. Multi-input Recurrent Independent Mechanisms for leveraging knowledge sources: Case studies on sentiment analysis and health text mining. In Proceedings of Deep Learning Inside Out (DeeLIO): The 2nd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures, pages 108–118, Online. Association for Computational Linguistics.
- Cite (Informal):
- Multi-input Recurrent Independent Mechanisms for leveraging knowledge sources: Case studies on sentiment analysis and health text mining (Bagherzadeh & Bergler, DeeLIO 2021)
- PDF:
- https://preview.aclanthology.org/ingest-acl-2023-videos/2021.deelio-1.11.pdf
- Data
- GLUE, SST, SST-2, TweetEval