Abstract
The task of scholarly keyphrase boundary classification aims at identifying keyphrases from scientific papers and classifying them with their types from a set of predefined classes (e.g., task, process, or material). Despite the importance of keyphrases and their types in many downstream applications including indexing, searching, and question answering over scientific documents, scholarly keyphrase boundary classification is still an under-explored task. In this work, we propose a novel embedding constraint on multi-task knowledge distillation which enforces the teachers (single-task models) and the student (multi-task model) similarity in the embedding space. Specifically, we enforce that the student model is trained not only to imitate the teachers’ output distribution over classes, but also to produce language representations that are similar to those produced by the teachers. Our results show that the proposed approach outperforms previous works and strong baselines on three datasets of scientific documents.- Anthology ID:
- 2023.emnlp-main.805
- Volume:
- Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
- Month:
- December
- Year:
- 2023
- Address:
- Singapore
- Editors:
- Houda Bouamor, Juan Pino, Kalika Bali
- Venue:
- EMNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 13026–13042
- Language:
- URL:
- https://aclanthology.org/2023.emnlp-main.805
- DOI:
- 10.18653/v1/2023.emnlp-main.805
- Cite (ACL):
- Seo Park and Cornelia Caragea. 2023. Multi-Task Knowledge Distillation with Embedding Constraints for Scholarly Keyphrase Boundary Classification. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 13026–13042, Singapore. Association for Computational Linguistics.
- Cite (Informal):
- Multi-Task Knowledge Distillation with Embedding Constraints for Scholarly Keyphrase Boundary Classification (Park & Caragea, EMNLP 2023)
- PDF:
- https://preview.aclanthology.org/corrections-2024-05/2023.emnlp-main.805.pdf