Can dependency parses facilitate generalization in language models? A case study of cross-lingual relation extraction

Ritam Dutt, Shounak Sural, Carolyn Rose


Abstract
In this work, we propose DEPGEN, a framework for evaluating the generalization capabilities of language models on the task of relation extraction, with dependency parses as scaffolds. We use a GNN-based framework that takes dependency parses as input and learns embeddings of entities which are augmented to a baseline multilingual encoder. We also investigate the role of dependency parses when they are included as part of the prompt to LLMs in a zero-shot learning setup. We observe that including off-the-shelf dependency parses can aid relation extraction, with the best performing model having a mild relative improvement of 0.91% and 1.5% in the in-domain and zero-shot setting respectively across two datasets. For the in-context learning setup, we observe an average improvement of 1.67%, with significant gains for low-performing LLMs. We also carry out extensive statistical analysis to investigate how different factors such as the choice of the dependency parser or the nature of the prompt impact performance. We make our code and results publicly available for the research community at https://github.com/ShoRit/multilingual-re.git.
Anthology ID:
2025.knowledgenlp-1.27
Volume:
Proceedings of the 4th International Workshop on Knowledge-Augmented Methods for Natural Language Processing
Month:
May
Year:
2025
Address:
Albuquerque, New Mexico, USA
Editors:
Weijia Shi, Wenhao Yu, Akari Asai, Meng Jiang, Greg Durrett, Hannaneh Hajishirzi, Luke Zettlemoyer
Venues:
KnowledgeNLP | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
317–337
Language:
URL:
https://preview.aclanthology.org/fix-sig-urls/2025.knowledgenlp-1.27/
DOI:
Bibkey:
Cite (ACL):
Ritam Dutt, Shounak Sural, and Carolyn Rose. 2025. Can dependency parses facilitate generalization in language models? A case study of cross-lingual relation extraction. In Proceedings of the 4th International Workshop on Knowledge-Augmented Methods for Natural Language Processing, pages 317–337, Albuquerque, New Mexico, USA. Association for Computational Linguistics.
Cite (Informal):
Can dependency parses facilitate generalization in language models? A case study of cross-lingual relation extraction (Dutt et al., KnowledgeNLP 2025)
Copy Citation:
PDF:
https://preview.aclanthology.org/fix-sig-urls/2025.knowledgenlp-1.27.pdf