A Unified Knowledge Graph Augmentation Service for Boosting Domain-specific NLP Tasks

Ruiqing Ding, Xiao Han, Leye Wang

Abstract
By focusing the pre-training process on domain-specific corpora, some domain-specific pre-trained language models (PLMs) have achieved state-of-the-art results. However, it is under-investigated to design a unified paradigm to inject domain knowledge in the PLM fine-tuning stage. We propose KnowledgeDA, a unified domain language model development service to enhance the task-specific training procedure with domain knowledge graphs. Given domain-specific task texts input, KnowledgeDA can automatically generate a domain-specific language model following three steps: (i) localize domain knowledge entities in texts via an embedding-similarity approach; (ii) generate augmented samples by retrieving replaceable domain entity pairs from two views of both knowledge graph and training data; (iii) select high-quality augmented samples for fine-tuning via confidence-based assessment. We implement a prototype of KnowledgeDA to learn language models for two domains, healthcare and software development. Experiments on domain-specific text classification and QA tasks verify the effectiveness and generalizability of KnowledgeDA.
Anthology ID:
2023.findings-acl.24
Volume:
Findings of the Association for Computational Linguistics: ACL 2023
Month:
July
Year:
2023
Address:
Toronto, Canada
Editors:
Anna Rogers, Jordan Boyd-Graber, Naoaki Okazaki
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
353–369
Language:
URL:
https://aclanthology.org/2023.findings-acl.24
DOI:
10.18653/v1/2023.findings-acl.24
Bibkey:
Cite (ACL):
Ruiqing Ding, Xiao Han, and Leye Wang. 2023. A Unified Knowledge Graph Augmentation Service for Boosting Domain-specific NLP Tasks. In Findings of the Association for Computational Linguistics: ACL 2023, pages 353–369, Toronto, Canada. Association for Computational Linguistics.
Cite (Informal):
A Unified Knowledge Graph Augmentation Service for Boosting Domain-specific NLP Tasks (Ding et al., Findings 2023)
Copy Citation:
PDF:
https://preview.aclanthology.org/nschneid-patch-5/2023.findings-acl.24.pdf
PDF Search