Abstract
Traditional keyphrase prediction methods predict a single set of keyphrases per document, failing to cater to the diverse needs of users and downstream applications. To bridge the gap, we introduce on-demand keyphrase generation, a novel paradigm that requires keyphrases that conform to specific high-level goals or intents. For this task, we present MetaKP, a large-scale benchmark comprising four datasets, 7500 documents, and 3760 goals across news and biomedical domains with human-annotated keyphrases. Leveraging MetaKP, we design both supervised and unsupervised methods, including a multi-task fine-tuning approach and a self-consistency prompting method with large language models. The results highlight the challenges of supervised fine-tuning, whose performance is not robust to distribution shifts. By contrast, the proposed self-consistency prompting approach greatly improves the performance of large language models, enabling GPT-4o to achieve 0.548 SemF1, surpassing the performance of a fully fine-tuned BART-base model. Finally, we demonstrate the potential of our method to serve as a general NLP infrastructure, exemplified by its application in epidemic event detection from social media.- Anthology ID:
- 2024.findings-emnlp.494
- Volume:
- Findings of the Association for Computational Linguistics: EMNLP 2024
- Month:
- November
- Year:
- 2024
- Address:
- Miami, Florida, USA
- Editors:
- Yaser Al-Onaizan, Mohit Bansal, Yun-Nung Chen
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 8420–8437
- Language:
- URL:
- https://preview.aclanthology.org/add_missing_videos/2024.findings-emnlp.494/
- DOI:
- 10.18653/v1/2024.findings-emnlp.494
- Cite (ACL):
- Di Wu, Xiaoxian Shen, and Kai-Wei Chang. 2024. MetaKP: On-Demand Keyphrase Generation. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 8420–8437, Miami, Florida, USA. Association for Computational Linguistics.
- Cite (Informal):
- MetaKP: On-Demand Keyphrase Generation (Wu et al., Findings 2024)
- PDF:
- https://preview.aclanthology.org/add_missing_videos/2024.findings-emnlp.494.pdf