Using Collostructional Analysis to evaluate BERT’s representation of linguistic constructions

Tim Veenboer, Jelke Bloem

Abstract
Collostructional analysis is a technique devised to find correlations between particular words and linguistic constructions in order to analyse meaning associations of these constructions. Contrasting collostructional analysis results with output from BERT might provide insights into the way BERT represents the meaning of linguistic constructions. This study tests to what extent English BERT’s meaning representations correspond to known constructions from the linguistics literature by means of two tasks that we propose. Firstly, by predicting the words that can be used in open slots of constructions, the meaning associations of more lexicalized constructions can be observed. Secondly, by finding similar sequences using BERT’s output embeddings and manually reviewing the resulting sentences, we can observe whether instances of less lexicalized constructions are clustered together in semantic space. These two methods show that BERT represents constructional meaning to a certain extent, but does not separate instances of a construction from a near-synonymous construction that has a different form.
Anthology ID:
2023.findings-acl.819
Volume:
Findings of the Association for Computational Linguistics: ACL 2023
Month:
July
Year:
2023
Address:
Toronto, Canada
Editors:
Anna Rogers, Jordan Boyd-Graber, Naoaki Okazaki
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
12937–12951
Language:
URL:
https://aclanthology.org/2023.findings-acl.819
DOI:
10.18653/v1/2023.findings-acl.819
Bibkey:
Cite (ACL):
Tim Veenboer and Jelke Bloem. 2023. Using Collostructional Analysis to evaluate BERT’s representation of linguistic constructions. In Findings of the Association for Computational Linguistics: ACL 2023, pages 12937–12951, Toronto, Canada. Association for Computational Linguistics.
Cite (Informal):
Using Collostructional Analysis to evaluate BERT’s representation of linguistic constructions (Veenboer & Bloem, Findings 2023)
Copy Citation:
PDF:
https://preview.aclanthology.org/emnlp22-frontmatter/2023.findings-acl.819.pdf
PDF Search