Evaluating the Development of Linguistic Metaphor Annotation in Mexican Spanish Popular Science Tweets
Alec Montero, Gemma Bel-Enguix, Sergio-Luis Ojeda-Trueba, Marisela Colín Rodea
Abstract
Following previous work on metaphor annotation and automatic metaphor processing, this study presents the evaluation of an initial phase in the novel area of linguistic metaphor detection in Mexican Spanish popular science tweets. Specifically, we examine the challenges posed by the annotation process stemming from disagreement among annotators. During this phase of our work, we conducted the annotation of a corpus comprising 3733 Mexican Spanish popular science tweets. This corpus was divided into two halves and each half was then assigned to two different pairs of native Mexican Spanish-speaking annotators. Despite rigorous methodology and continuous training, inter-annotator agreement as measured by Cohen’s kappa was found to be low, slightly above chance levels, although the concordance percentage exceeded 60%. By elucidating the inherent complexity of metaphor annotation tasks, our evaluation emphasizes the implications of these findings and offers insights for future research in this field, with the aim of creating a robust dataset for machine learning.- Anthology ID:
- 2024.figlang-1.8
- Volume:
- Proceedings of the 4th Workshop on Figurative Language Processing (FigLang 2024)
- Month:
- June
- Year:
- 2024
- Address:
- Mexico City, Mexico (Hybrid)
- Editors:
- Debanjan Ghosh, Smaranda Muresan, Anna Feldman, Tuhin Chakrabarty, Emmy Liu
- Venues:
- Fig-Lang | WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 59–64
- Language:
- URL:
- https://aclanthology.org/2024.figlang-1.8
- DOI:
- Cite (ACL):
- Alec Montero, Gemma Bel-Enguix, Sergio-Luis Ojeda-Trueba, and Marisela Colín Rodea. 2024. Evaluating the Development of Linguistic Metaphor Annotation in Mexican Spanish Popular Science Tweets. In Proceedings of the 4th Workshop on Figurative Language Processing (FigLang 2024), pages 59–64, Mexico City, Mexico (Hybrid). Association for Computational Linguistics.
- Cite (Informal):
- Evaluating the Development of Linguistic Metaphor Annotation in Mexican Spanish Popular Science Tweets (Montero et al., Fig-Lang-WS 2024)
- PDF:
- https://preview.aclanthology.org/jeptaln-2024-ingestion/2024.figlang-1.8.pdf