ABRICOT - ABstRactness and Inclusiveness in COntexT: A CALAMITA Challenge
Giovanni Puccetti, Claudia Collacciani, Andrea Amelio Ravelli, Andrea Esuli, Marianna Bolognesi
Abstract
The ABRICOT Task is designed to evaluate Italian language models on their ability to understand and assess the abstractness and inclusiveness of language, two nuanced features that humans naturally convey in everyday communication. Unlike binary categorizations such as abstract/concrete or inclusive/exclusive, these features exist on a continuous spectrum with varying degrees of intensity. The task is based on a manual collection of sentences that present the same noun phrase (NP) in different contexts, allowing its interpretation to vary between the extremes of abstractness and inclusiveness. This challenge aims to verify the how LLMs perceive subtle linguistic variations and their implications in natural language.- Anthology ID:
- 2024.clicit-1.128
- Volume:
- Proceedings of the 10th Italian Conference on Computational Linguistics (CLiC-it 2024)
- Month:
- December
- Year:
- 2024
- Address:
- Pisa, Italy
- Editors:
- Felice Dell'Orletta, Alessandro Lenci, Simonetta Montemagni, Rachele Sprugnoli
- Venue:
- CLiC-it
- SIG:
- Publisher:
- CEUR Workshop Proceedings
- Note:
- Pages:
- 1161–1167
- Language:
- URL:
- https://preview.aclanthology.org/fix-sig-urls/2024.clicit-1.128/
- DOI:
- Cite (ACL):
- Giovanni Puccetti, Claudia Collacciani, Andrea Amelio Ravelli, Andrea Esuli, and Marianna Bolognesi. 2024. ABRICOT - ABstRactness and Inclusiveness in COntexT: A CALAMITA Challenge. In Proceedings of the 10th Italian Conference on Computational Linguistics (CLiC-it 2024), pages 1161–1167, Pisa, Italy. CEUR Workshop Proceedings.
- Cite (Informal):
- ABRICOT - ABstRactness and Inclusiveness in COntexT: A CALAMITA Challenge (Puccetti et al., CLiC-it 2024)
- PDF:
- https://preview.aclanthology.org/fix-sig-urls/2024.clicit-1.128.pdf