Abstract
Bi-encoder architectures for distantly-supervised relation extraction are designed to make use of the complementary information found in text and knowledge graphs (KG).However, current architectures suffer from two drawbacks. They either do not allow any sharing between the text encoder and the KG encoder at all, or, in case of models with KG-to-text attention, only share information in one direction. Here, we introduce cross-stitch bi-encoders, which allow full interaction between the text encoder and the KG encoder via a cross-stitch mechanism. The cross-stitch mechanism allows sharing and updating representations between the two encoders at any layer, with the amount of sharing being dynamically controlled via cross-attention-based gates. Experimental results on two relation extraction benchmarks from two different domains show that enabling full interaction between the two encoders yields strong improvements.- Anthology ID:
- 2022.emnlp-main.467
- Volume:
- Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing
- Month:
- December
- Year:
- 2022
- Address:
- Abu Dhabi, United Arab Emirates
- Editors:
- Yoav Goldberg, Zornitsa Kozareva, Yue Zhang
- Venue:
- EMNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 6947–6958
- Language:
- URL:
- https://aclanthology.org/2022.emnlp-main.467
- DOI:
- 10.18653/v1/2022.emnlp-main.467
- Cite (ACL):
- Qin Dai, Benjamin Heinzerling, and Kentaro Inui. 2022. Cross-stitching Text and Knowledge Graph Encoders for Distantly Supervised Relation Extraction. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 6947–6958, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
- Cite (Informal):
- Cross-stitching Text and Knowledge Graph Encoders for Distantly Supervised Relation Extraction (Dai et al., EMNLP 2022)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-2/2022.emnlp-main.467.pdf