A Unified Framework for Synaesthesia Analysis

Kun Sheng, Zhongqing Wang, Qingqing Zhao, Xiaotong Jiang, Guodong Zhou

Abstract
Synaesthesia refers to the description of perceptions in one sensory modality through concepts from other modalities. It involves not only a linguistic phenomenon, but also a cognitive phenomenon structuring human thought and action, which makes understanding it challenging. As a means of cognition, synaesthesia is rendered by more than sensory modalities, cue and stimulus can also play an important role in expressing and understanding it. In addition, understanding synaesthesia involves many cognitive efforts, such as identifying the semantic relationship between sensory words and modalities. Therefore, we propose a unified framework focusing on annotating all kinds of synaesthetic elements and fully exploring the relationship among them. In particular, we introduce a new annotation scheme, including sensory modalities as well as their cues and stimuli, which facilitate understanding synaesthetic information collectively. We further design a structure generation model to capture the relations among synaesthetic elements and generate them jointly. Through extensive experiments, the importance of proposed dataset can be verified by the statistics and progressive performances. In addition, our proposed model yields state-of-the-art results, demonstrating its effectiveness.
Anthology ID:
2023.findings-emnlp.401
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2023
Month:
December
Year:
2023
Address:
Singapore
Editors:
Houda Bouamor, Juan Pino, Kalika Bali
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
6038–6048
Language:
URL:
https://aclanthology.org/2023.findings-emnlp.401
DOI:
10.18653/v1/2023.findings-emnlp.401
Bibkey:
Cite (ACL):
Kun Sheng, Zhongqing Wang, Qingqing Zhao, Xiaotong Jiang, and Guodong Zhou. 2023. A Unified Framework for Synaesthesia Analysis. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 6038–6048, Singapore. Association for Computational Linguistics.
Cite (Informal):
A Unified Framework for Synaesthesia Analysis (Sheng et al., Findings 2023)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-2024-clasp/2023.findings-emnlp.401.pdf
PDF Search