Scalable Evaluation and Improvement of Document Set Expansion via Neural Positive-Unlabeled Learning
Abstract
We consider the situation in which a user has collected a small set of documents on a cohesive topic, and they want to retrieve additional documents on this topic from a large collection. Information Retrieval (IR) solutions treat the document set as a query, and look for similar documents in the collection. We propose to extend the IR approach by treating the problem as an instance of positive-unlabeled (PU) learning—i.e., learning binary classifiers from only positive (the query documents) and unlabeled (the results of the IR engine) data. Utilizing PU learning for text with big neural networks is a largely unexplored field. We discuss various challenges in applying PU learning to the setting, showing that the standard implementations of state-of-the-art PU solutions fail. We propose solutions for each of the challenges and empirically validate them with ablation tests. We demonstrate the effectiveness of the new method using a series of experiments of retrieving PubMed abstracts adhering to fine-grained topics, showing improvements over the common IR solution and other baselines.- Anthology ID:
- 2021.eacl-main.47
- Volume:
- Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume
- Month:
- April
- Year:
- 2021
- Address:
- Online
- Venue:
- EACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 581–592
- Language:
- URL:
- https://aclanthology.org/2021.eacl-main.47
- DOI:
- 10.18653/v1/2021.eacl-main.47
- Cite (ACL):
- Alon Jacovi, Gang Niu, Yoav Goldberg, and Masashi Sugiyama. 2021. Scalable Evaluation and Improvement of Document Set Expansion via Neural Positive-Unlabeled Learning. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 581–592, Online. Association for Computational Linguistics.
- Cite (Informal):
- Scalable Evaluation and Improvement of Document Set Expansion via Neural Positive-Unlabeled Learning (Jacovi et al., EACL 2021)
- PDF:
- https://preview.aclanthology.org/ingestion-script-update/2021.eacl-main.47.pdf
- Code
- sayaendo/document-set-expansion-pu