CogALex-V Shared Task: GHHH - Detecting Semantic Relations via Word Embeddings

Mohammed Attia, Suraj Maharjan, Younes Samih, Laura Kallmeyer, Thamar Solorio

Abstract
This paper describes our system submission to the CogALex-2016 Shared Task on Corpus-Based Identification of Semantic Relations. Our system won first place for Task-1 and second place for Task-2. The evaluation results of our system on the test set is 88.1% (79.0% for TRUE only) f-measure for Task-1 on detecting semantic similarity, and 76.0% (42.3% when excluding RANDOM) for Task-2 on identifying finer-grained semantic relations. In our experiments, we try word analogy, linear regression, and multi-task Convolutional Neural Networks (CNNs) with word embeddings from publicly available word vectors. We found that linear regression performs better in the binary classification (Task-1), while CNNs have better performance in the multi-class semantic classification (Task-2). We assume that word analogy is more suited for deterministic answers rather than handling the ambiguity of one-to-many and many-to-many relationships. We also show that classifier performance could benefit from balancing the distribution of labels in the training data.
Anthology ID:
W16-5311
Volume:
Proceedings of the 5th Workshop on Cognitive Aspects of the Lexicon (CogALex - V)
Month:
December
Year:
2016
Address:
Osaka, Japan
Editors:
Michael Zock, Alessandro Lenci, Stefan Evert
Venue:
CogALex
SIG:
SIGLEX
Publisher:
The COLING 2016 Organizing Committee
Note:
Pages:
86–91
Language:
URL:
https://aclanthology.org/W16-5311
DOI:
Bibkey:
Cite (ACL):
Mohammed Attia, Suraj Maharjan, Younes Samih, Laura Kallmeyer, and Thamar Solorio. 2016. CogALex-V Shared Task: GHHH - Detecting Semantic Relations via Word Embeddings. In Proceedings of the 5th Workshop on Cognitive Aspects of the Lexicon (CogALex - V), pages 86–91, Osaka, Japan. The COLING 2016 Organizing Committee.
Cite (Informal):
CogALex-V Shared Task: GHHH - Detecting Semantic Relations via Word Embeddings (Attia et al., CogALex 2016)
Copy Citation:
PDF:
https://preview.aclanthology.org/improve-issue-templates/W16-5311.pdf
PDF Search