Abstract
Non-Negative Randomized Word Embedding We propose a word embedding method which is based on a novel random projection technique. We show that weighting methods such as positive pointwise mutual information (PPMI) can be applied to our models after their construction and at a reduced dimensionality. Hence, the proposed technique can efficiently transfer words onto semantically discriminative spaces while demonstrating high computational performance, besides benefits such as ease of update and a simple mechanism for interoperability. We report the performance of our method on several tasks and show that it yields competitive results compared to neural embedding methods in monolingual corpus-based setups.- Anthology ID:
- 2017.jeptalnrecital-long.8
- Volume:
- Actes des 24ème Conférence sur le Traitement Automatique des Langues Naturelles. Volume 1 - Articles longs
- Month:
- 6
- Year:
- 2017
- Address:
- Orléans, France
- Editors:
- Iris Eshkol-Taravella, Jean-Yves Antoine
- Venue:
- JEP/TALN/RECITAL
- SIG:
- Publisher:
- ATALA
- Note:
- Pages:
- 109–122
- Language:
- URL:
- https://aclanthology.org/2017.jeptalnrecital-long.8
- DOI:
- Cite (ACL):
- Behrang Qasemizadeh, Laura Kallmeyer, and Aurelie Herbelot. 2017. Projection Aléatoire Non-Négative pour le Calcul de Word Embedding / Non-Negative Randomized Word Embedding. In Actes des 24ème Conférence sur le Traitement Automatique des Langues Naturelles. Volume 1 - Articles longs, pages 109–122, Orléans, France. ATALA.
- Cite (Informal):
- Projection Aléatoire Non-Négative pour le Calcul de Word Embedding / Non-Negative Randomized Word Embedding (Qasemizadeh et al., JEP/TALN/RECITAL 2017)
- PDF:
- https://preview.aclanthology.org/ingest-acl-2023-videos/2017.jeptalnrecital-long.8.pdf