The CogALex-V Shared Task on the Corpus-Based Identification of Semantic Relations
Enrico Santus, Anna Gladkova, Stefan Evert, Alessandro Lenci
Abstract
The shared task of the 5th Workshop on Cognitive Aspects of the Lexicon (CogALex-V) aims at providing a common benchmark for testing current corpus-based methods for the identification of lexical semantic relations (synonymy, antonymy, hypernymy, part-whole meronymy) and at gaining a better understanding of their respective strengths and weaknesses. The shared task uses a challenging dataset extracted from EVALution 1.0, which contains word pairs holding the above-mentioned relations as well as semantically unrelated control items (random). The task is split into two subtasks: (i) identification of related word pairs vs. unrelated ones; (ii) classification of the word pairs according to their semantic relation. This paper describes the subtasks, the dataset, the evaluation metrics, the seven participating systems and their results. The best performing system in subtask 1 is GHHH (F1 = 0.790), while the best system in subtask 2 is LexNet (F1 = 0.445). The dataset and the task description are available at https://sites.google.com/site/cogalex2016/home/shared-task.- Anthology ID:
- W16-5309
- Volume:
- Proceedings of the 5th Workshop on Cognitive Aspects of the Lexicon (CogALex - V)
- Month:
- December
- Year:
- 2016
- Address:
- Osaka, Japan
- Editors:
- Michael Zock, Alessandro Lenci, Stefan Evert
- Venue:
- CogALex
- SIG:
- SIGLEX
- Publisher:
- The COLING 2016 Organizing Committee
- Note:
- Pages:
- 69–79
- Language:
- URL:
- https://aclanthology.org/W16-5309
- DOI:
- Cite (ACL):
- Enrico Santus, Anna Gladkova, Stefan Evert, and Alessandro Lenci. 2016. The CogALex-V Shared Task on the Corpus-Based Identification of Semantic Relations. In Proceedings of the 5th Workshop on Cognitive Aspects of the Lexicon (CogALex - V), pages 69–79, Osaka, Japan. The COLING 2016 Organizing Committee.
- Cite (Informal):
- The CogALex-V Shared Task on the Corpus-Based Identification of Semantic Relations (Santus et al., CogALex 2016)
- PDF:
- https://preview.aclanthology.org/dois-2013-emnlp/W16-5309.pdf
- Data
- EVALution