Quantifying the morphosyntactic content of Brown Clusters

Manuel R. Ciosici, Leon Derczynski, Ira Assent


Abstract
Brown and Exchange word clusters have long been successfully used as word representations in Natural Language Processing (NLP) systems. Their success has been attributed to their seeming ability to represent both semantic and syntactic information. Using corpora representing several language families, we test the hypothesis that Brown and Exchange word clusters are highly effective at encoding morphosyntactic information. Our experiments show that word clusters are highly capable at distinguishing Parts of Speech. We show that increases in Average Mutual Information, the clustering algorithms’ optimization goal, are highly correlated with improvements in encoding of morphosyntactic information. Our results provide empirical evidence that downstream NLP systems addressing tasks dependent on morphosyntactic information can benefit from word cluster features.
Anthology ID:
N19-1157
Volume:
Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)
Month:
June
Year:
2019
Address:
Minneapolis, Minnesota
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1541–1550
Language:
URL:
https://aclanthology.org/N19-1157
DOI:
10.18653/v1/N19-1157
Bibkey:
Cite (ACL):
Manuel R. Ciosici, Leon Derczynski, and Ira Assent. 2019. Quantifying the morphosyntactic content of Brown Clusters. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 1541–1550, Minneapolis, Minnesota. Association for Computational Linguistics.
Cite (Informal):
Quantifying the morphosyntactic content of Brown Clusters (Ciosici et al., NAACL 2019)
Copy Citation:
PDF:
https://preview.aclanthology.org/update-css-js/N19-1157.pdf
Supplementary:
 N19-1157.Supplementary.pdf
Video:
 https://vimeo.com/360726205