CALaMo: a Constructionist Assessment of Language Models

Ludovica Pannitto, Aurélie Herbelot


Abstract
This paper presents a novel framework for evaluating Neural Language Models’ linguistic abilities using a constructionist approach. Not only is the usage-based model in line with the un- derlying stochastic philosophy of neural architectures, but it also allows the linguist to keep meaning as a determinant factor in the analysis. We outline the framework and present two possible scenarios for its application.
Anthology ID:
2023.cxgsnlp-1.3
Volume:
Proceedings of the First International Workshop on Construction Grammars and NLP (CxGs+NLP, GURT/SyntaxFest 2023)
Month:
March
Year:
2023
Address:
Washington, D.C.
Venues:
CxGsNLP | SyntaxFest
SIG:
SIGPARSE
Publisher:
Association for Computational Linguistics
Note:
Pages:
21–30
Language:
URL:
https://aclanthology.org/2023.cxgsnlp-1.3
DOI:
Bibkey:
Cite (ACL):
Ludovica Pannitto and Aurélie Herbelot. 2023. CALaMo: a Constructionist Assessment of Language Models. In Proceedings of the First International Workshop on Construction Grammars and NLP (CxGs+NLP, GURT/SyntaxFest 2023), pages 21–30, Washington, D.C.. Association for Computational Linguistics.
Cite (Informal):
CALaMo: a Constructionist Assessment of Language Models (Pannitto & Herbelot, CxGsNLP-SyntaxFest 2023)
Copy Citation:
PDF:
https://preview.aclanthology.org/remove-xml-comments/2023.cxgsnlp-1.3.pdf
Video:
 https://preview.aclanthology.org/remove-xml-comments/2023.cxgsnlp-1.3.mov