Abstract
Many types of distributional word embeddings (weakly) encode linguistic regularities as directions (the difference between jump and jumped will be in a similar direction to that of walk and walked, and so on). Several attempts have been made to explain this fact. We respond to Allen and Hospedales’ recent (ICML, 2019) theoretical explanation, which claims that word2vec and GloVe will encode linguistic regularities whenever a specific relation of paraphrase holds between the four words involved in the regularity. We demonstrate that the explanation does not go through: the paraphrase relations needed under this explanation do not hold empirically- Anthology ID:
- 2021.eacl-main.182
- Volume:
- Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume
- Month:
- April
- Year:
- 2021
- Address:
- Online
- Venue:
- EACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 2129–2134
- Language:
- URL:
- https://aclanthology.org/2021.eacl-main.182
- DOI:
- 10.18653/v1/2021.eacl-main.182
- Cite (ACL):
- Louis Fournier and Ewan Dunbar. 2021. Paraphrases do not explain word analogies. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 2129–2134, Online. Association for Computational Linguistics.
- Cite (Informal):
- Paraphrases do not explain word analogies (Fournier & Dunbar, EACL 2021)
- PDF:
- https://preview.aclanthology.org/ingestion-script-update/2021.eacl-main.182.pdf
- Code
- bootphon/paraphrases_do_not_explain_analogies