Abstract
Recently Le & Mikolov described two log-linear models, called Paragraph Vector, that can be used to learn state-of-the-art distributed representations of documents. Inspired by this work, we present Binary Paragraph Vector models: simple neural networks that learn short binary codes for fast information retrieval. We show that binary paragraph vectors outperform autoencoder-based binary codes, despite using fewer bits. We also evaluate their precision in transfer learning settings, where binary codes are inferred for documents unrelated to the training corpus. Results from these experiments indicate that binary paragraph vectors can capture semantics relevant for various domain-specific documents. Finally, we present a model that simultaneously learns short binary codes and longer, real-valued representations. This model can be used to rapidly retrieve a short list of highly relevant documents from a large document collection.- Anthology ID:
- W17-2615
- Volume:
- Proceedings of the 2nd Workshop on Representation Learning for NLP
- Month:
- August
- Year:
- 2017
- Address:
- Vancouver, Canada
- Editors:
- Phil Blunsom, Antoine Bordes, Kyunghyun Cho, Shay Cohen, Chris Dyer, Edward Grefenstette, Karl Moritz Hermann, Laura Rimell, Jason Weston, Scott Yih
- Venue:
- RepL4NLP
- SIG:
- SIGREP
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 121–130
- Language:
- URL:
- https://aclanthology.org/W17-2615
- DOI:
- 10.18653/v1/W17-2615
- Cite (ACL):
- Karol Grzegorczyk and Marcin Kurdziel. 2017. Binary Paragraph Vectors. In Proceedings of the 2nd Workshop on Representation Learning for NLP, pages 121–130, Vancouver, Canada. Association for Computational Linguistics.
- Cite (Informal):
- Binary Paragraph Vectors (Grzegorczyk & Kurdziel, RepL4NLP 2017)
- PDF:
- https://preview.aclanthology.org/ingest-acl-2023-videos/W17-2615.pdf