@inproceedings{zhou-etal-2019-bert,
    title = "{BERT}-based Lexical Substitution",
    author = "Zhou, Wangchunshu  and
      Ge, Tao  and
      Xu, Ke  and
      Wei, Furu  and
      Zhou, Ming",
    editor = "Korhonen, Anna  and
      Traum, David  and
      M{\`a}rquez, Llu{\'i}s",
    booktitle = "Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics",
    month = jul,
    year = "2019",
    address = "Florence, Italy",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/add-emnlp-2024-awards/P19-1328/",
    doi = "10.18653/v1/P19-1328",
    pages = "3368--3373",
    abstract = "Previous studies on lexical substitution tend to obtain substitute candidates by finding the target word`s synonyms from lexical resources (e.g., WordNet) and then rank the candidates based on its contexts. These approaches have two limitations: (1) They are likely to overlook good substitute candidates that are not the synonyms of the target words in the lexical resources; (2) They fail to take into account the substitution`s influence on the global context of the sentence. To address these issues, we propose an end-to-end BERT-based lexical substitution approach which can propose and validate substitute candidates without using any annotated data or manually curated resources. Our approach first applies dropout to the target word`s embedding for partially masking the word, allowing BERT to take balanced consideration of the target word`s semantics and contexts for proposing substitute candidates, and then validates the candidates based on their substitution`s influence on the global contextualized representation of the sentence. Experiments show our approach performs well in both proposing and ranking substitute candidates, achieving the state-of-the-art results in both LS07 and LS14 benchmarks."
}