Abstract
This study addresses the interpretability of word representations through an investigation of a count-based co-occurrence matrix. Employing the mathematical methodology of Formal Concept Analysis, we reveal an underlying structure that is amenable to human interpretation. Furthermore, we unveil the emergence of hierarchical and geometrical structures within word vectors as consequences of word usage. Our experiments on the PPMI matrix demonstrate that the formal concepts that we identified align with interpretable categories, as shown in the category completion task.- Anthology ID:
- 2024.findings-acl.278
- Volume:
- Findings of the Association for Computational Linguistics ACL 2024
- Month:
- August
- Year:
- 2024
- Address:
- Bangkok, Thailand and virtual meeting
- Editors:
- Lun-Wei Ku, Andre Martins, Vivek Srikumar
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 4683–4700
- Language:
- URL:
- https://aclanthology.org/2024.findings-acl.278
- DOI:
- Cite (ACL):
- Akihiro Maeda, Takuma Torii, and Shohei Hidaka. 2024. Decomposing Co-occurrence Matrices into Interpretable Components as Formal Concepts. In Findings of the Association for Computational Linguistics ACL 2024, pages 4683–4700, Bangkok, Thailand and virtual meeting. Association for Computational Linguistics.
- Cite (Informal):
- Decomposing Co-occurrence Matrices into Interpretable Components as Formal Concepts (Maeda et al., Findings 2024)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-4/2024.findings-acl.278.pdf