Abstract
Idiomatic expressions (IEs), characterized by their non-compositionality, are an important part of natural language. They have been a classical challenge to NLP, including pre-trained language models that drive today’s state-of-the-art. Prior work has identified deficiencies in their contextualized representation stemming from the underlying compositional paradigm of representation. In this work, we take a first-principles approach to build idiomaticity into BART using an adapter as a lightweight non-compositional language expert trained on idiomatic sentences. The improved capability over baselines (e.g., BART) is seen via intrinsic and extrinsic methods, where idiom embeddings score 0.19 points higher in homogeneity score for embedding clustering, and up to 25% higher sequence accuracy on the idiom processing tasks of IE sense disambiguation and span detection.- Anthology ID:
- 2022.tacl-1.65
- Volume:
- Transactions of the Association for Computational Linguistics, Volume 10
- Month:
- Year:
- 2022
- Address:
- Cambridge, MA
- Editors:
- Brian Roark, Ani Nenkova
- Venue:
- TACL
- SIG:
- Publisher:
- MIT Press
- Note:
- Pages:
- 1120–1137
- Language:
- URL:
- https://aclanthology.org/2022.tacl-1.65
- DOI:
- 10.1162/tacl_a_00510
- Cite (ACL):
- Ziheng Zeng and Suma Bhat. 2022. Getting BART to Ride the Idiomatic Train: Learning to Represent Idiomatic Expressions. Transactions of the Association for Computational Linguistics, 10:1120–1137.
- Cite (Informal):
- Getting BART to Ride the Idiomatic Train: Learning to Represent Idiomatic Expressions (Zeng & Bhat, TACL 2022)
- PDF:
- https://preview.aclanthology.org/proper-vol2-ingestion/2022.tacl-1.65.pdf