@inproceedings{saha-etal-2022-similarity,
    title = "Similarity Based Label Smoothing For Dialogue Generation",
    author = "Saha, Sougata  and
      Das, Souvik  and
      Srihari, Rohini",
    editor = "Akhtar, Md. Shad  and
      Chakraborty, Tanmoy",
    booktitle = "Proceedings of the 19th International Conference on Natural Language Processing (ICON)",
    month = dec,
    year = "2022",
    address = "New Delhi, India",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/add-emnlp-2024-awards/2022.icon-main.31/",
    pages = "253--259",
    abstract = "Generative neural conversational systems are typically trained by minimizing the entropy loss between the training {\textquotedblleft}hard{\textquotedblright} targets and the predicted logits. Performance gains and improved generalization are often achieved by employing regularization techniques like label smoothing, which converts the training {\textquotedblleft}hard{\textquotedblright} targets to soft targets. However, label smoothing enforces a data independent uniform distribution on the incorrect training targets, leading to a false assumption of equiprobability. In this paper, we propose and experiment with incorporating data-dependent word similarity-based weighing methods to transform the uniform distribution of the incorrect target probabilities in label smoothing to a more realistic distribution based on semantics. We introduce hyperparameters to control the incorrect target distribution and report significant performance gains over networks trained using standard label smoothing-based loss on two standard open-domain dialogue corpora."
}