@inproceedings{nicolai-silfverberg-2020-noise,
    title = "Noise Isn`t Always Negative: Countering Exposure Bias in Sequence-to-Sequence Inflection Models",
    author = "Nicolai, Garrett  and
      Silfverberg, Miikka",
    editor = "Scott, Donia  and
      Bel, Nuria  and
      Zong, Chengqing",
    booktitle = "Proceedings of the 28th International Conference on Computational Linguistics",
    month = dec,
    year = "2020",
    address = "Barcelona, Spain (Online)",
    publisher = "International Committee on Computational Linguistics",
    url = "https://preview.aclanthology.org/Author-page-Marten-During-lu/2020.coling-main.255/",
    doi = "10.18653/v1/2020.coling-main.255",
    pages = "2837--2846",
    abstract = "Morphological inflection, like many sequence-to-sequence tasks, sees great performance from recurrent neural architectures when data is plentiful, but performance falls off sharply in lower-data settings. We investigate one aspect of neural seq2seq models that we hypothesize contributes to overfitting - teacher forcing. By creating different training and test conditions, exposure bias increases the likelihood that a system too closely models its training data. Experiments show that teacher-forced models struggle to recover when they enter unknown territory. However, a simple modification to the training algorithm to more closely mimic test conditions creates models that are better able to generalize to unseen environments."
}