Abstract
Representation learning forms an essential building block in the development of natural language processing architectures. To date, mainstream approaches focus on learning textual information at the sentence- or document-level, unfortunately, overlooking the inter-document connections. This omission decreases the potency of downstream applications, particularly in multi-document settings. To address this issue, embeddings equipped with latent semantic and rich relatedness information are needed. In this paper, we propose SMRC2, which extends representation learning to the multi-document level. Our model jointly learns latent semantic information from content and rich relatedness information from topological networks. Unlike previous studies, our work takes multi-document as input and integrates both semantic and relatedness information using a shared space via language model and graph structure. Our extensive experiments confirm the superiority and effectiveness of our approach. To encourage further research in scientific multi-literature representation learning, we will release our code and a new dataset from the biomedical domain.- Anthology ID:
- 2023.emnlp-main.465
- Volume:
- Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
- Month:
- December
- Year:
- 2023
- Address:
- Singapore
- Editors:
- Houda Bouamor, Juan Pino, Kalika Bali
- Venue:
- EMNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 7490–7502
- Language:
- URL:
- https://aclanthology.org/2023.emnlp-main.465
- DOI:
- 10.18653/v1/2023.emnlp-main.465
- Cite (ACL):
- Kai Zhang, Kaisong Song, Yangyang Kang, and Xiaozhong Liu. 2023. Content- and Topology-Aware Representation Learning for Scientific Multi-Literature. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 7490–7502, Singapore. Association for Computational Linguistics.
- Cite (Informal):
- Content- and Topology-Aware Representation Learning for Scientific Multi-Literature (Zhang et al., EMNLP 2023)
- PDF:
- https://preview.aclanthology.org/landing_page/2023.emnlp-main.465.pdf