Semantics or spelling? Probing contextual word embeddings with orthographic noise

Jacob Matthews, John Starr, Marten Schijndel

Abstract
Pretrained language model (PLM) hidden states are frequently employed as contextual word embeddings (CWE): high-dimensional representations that encode semantic information given linguistic context. Across many areas of computational linguistics research, similarity between CWEs is interpreted as semantic similarity. However, it remains unclear exactly what information is encoded in PLM hidden states. We investigate this practice by probing PLM representations using minimal orthographic noise. We expect that if CWEs primarily encode semantic information, a single character swap in the input word will not drastically affect the resulting representation, given sufficient linguistic context. Surprisingly, we find that CWEs generated by popular PLMs are highly sensitive to noise in input data, and that this sensitivity is related to subword tokenization: the fewer tokens used to represent a word at input, the more sensitive its corresponding CWE. This suggests that CWEs capture information unrelated to word-level meaning and can be manipulated through trivial modifications of input data. We conclude that these PLM-derived CWEs may not be reliable semantic proxies, and that caution is warranted when interpreting representational similarity.
Anthology ID:
2024.findings-acl.266
Volume:
Findings of the Association for Computational Linguistics ACL 2024
Month:
August
Year:
2024
Address:
Bangkok, Thailand and virtual meeting
Editors:
Lun-Wei Ku, Andre Martins, Vivek Srikumar
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
4495–4504
Language:
URL:
https://aclanthology.org/2024.findings-acl.266
DOI:
Bibkey:
Cite (ACL):
Jacob Matthews, John Starr, and Marten Schijndel. 2024. Semantics or spelling? Probing contextual word embeddings with orthographic noise. In Findings of the Association for Computational Linguistics ACL 2024, pages 4495–4504, Bangkok, Thailand and virtual meeting. Association for Computational Linguistics.
Cite (Informal):
Semantics or spelling? Probing contextual word embeddings with orthographic noise (Matthews et al., Findings 2024)
Copy Citation:
PDF:
https://preview.aclanthology.org/nschneid-patch-4/2024.findings-acl.266.pdf
PDF Search