Variation and generality in encoding of syntactic anomaly information in sentence embeddings

Qinxuan Wu, Allyson Ettinger

Abstract
While sentence anomalies have been applied periodically for testing in NLP, we have yet to establish a picture of the precise status of anomaly information in representations from NLP models. In this paper we aim to fill two primary gaps, focusing on the domain of syntactic anomalies. First, we explore fine-grained differences in anomaly encoding by designing probing tasks that vary the hierarchical level at which anomalies occur in a sentence. Second, we test not only models’ ability to detect a given anomaly, but also the generality of the detected anomaly signal, by examining transfer between distinct anomaly types. Results suggest that all models encode some information supporting anomaly detection, but detection performance varies between anomalies, and only representations from more re- cent transformer models show signs of generalized knowledge of anomalies. Follow-up analyses support the notion that these models pick up on a legitimate, general notion of sentence oddity, while coarser-grained word position information is likely also a contributor to the observed anomaly detection.
Anthology ID:
2021.blackboxnlp-1.18
Volume:
Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP
Month:
November
Year:
2021
Address:
Punta Cana, Dominican Republic
Editors:
Jasmijn Bastings, Yonatan Belinkov, Emmanuel Dupoux, Mario Giulianelli, Dieuwke Hupkes, Yuval Pinter, Hassan Sajjad
Venue:
BlackboxNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
250–264
Language:
URL:
https://aclanthology.org/2021.blackboxnlp-1.18
DOI:
10.18653/v1/2021.blackboxnlp-1.18
Bibkey:
Cite (ACL):
Qinxuan Wu and Allyson Ettinger. 2021. Variation and generality in encoding of syntactic anomaly information in sentence embeddings. In Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, pages 250–264, Punta Cana, Dominican Republic. Association for Computational Linguistics.
Cite (Informal):
Variation and generality in encoding of syntactic anomaly information in sentence embeddings (Wu & Ettinger, BlackboxNLP 2021)
Copy Citation:
PDF:
https://preview.aclanthology.org/nschneid-patch-2/2021.blackboxnlp-1.18.pdf
Data
CoLA
PDF Search