@inproceedings{takahashi-etal-2022-leveraging,
    title = "Leveraging Three Types of Embeddings from Masked Language Models in Idiom Token Classification",
    author = "Takahashi, Ryosuke  and
      Sasano, Ryohei  and
      Takeda, Koichi",
    editor = "Nastase, Vivi  and
      Pavlick, Ellie  and
      Pilehvar, Mohammad Taher  and
      Camacho-Collados, Jose  and
      Raganato, Alessandro",
    booktitle = "Proceedings of the 11th Joint Conference on Lexical and Computational Semantics",
    month = jul,
    year = "2022",
    address = "Seattle, Washington",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/fix-sig-urls/2022.starsem-1.21/",
    doi = "10.18653/v1/2022.starsem-1.21",
    pages = "234--239",
    abstract = "Many linguistic expressions have idiomatic and literal interpretations, and the automatic distinction of these two interpretations has been studied for decades. Recent research has shown that contextualized word embeddings derived from masked language models (MLMs) can give promising results for idiom token classification. This indicates that contextualized word embedding alone contains information about whether the word is being used in a literal sense or not. However, we believe that more types of information can be derived from MLMs and that leveraging such information can improve idiom token classification. In this paper, we leverage three types of embeddings from MLMs; uncontextualized token embeddings and masked token embeddings in addition to the standard contextualized word embeddings and show that the newly added embeddings significantly improve idiom token classification for both English and Japanese datasets."
}