@inproceedings{sengupta-etal-2021-robustness,
    title = "On the Robustness of Intent Classification and Slot Labeling in Goal-oriented Dialog Systems to Real-world Noise",
    author = "Sengupta, Sailik  and
      Krone, Jason  and
      Mansour, Saab",
    editor = "Papangelis, Alexandros  and
      Budzianowski, Pawe{\l}  and
      Liu, Bing  and
      Nouri, Elnaz  and
      Rastogi, Abhinav  and
      Chen, Yun-Nung",
    booktitle = "Proceedings of the 3rd Workshop on Natural Language Processing for Conversational AI",
    month = nov,
    year = "2021",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/fix-sig-urls/2021.nlp4convai-1.7/",
    doi = "10.18653/v1/2021.nlp4convai-1.7",
    pages = "68--79",
    abstract = "Intent Classification (IC) and Slot Labeling (SL) models, which form the basis of dialogue systems, often encounter noisy data in real-word environments. In this work, we investigate how robust IC/SL models are to noisy data. We collect and publicly release a test-suite for seven common noise types found in production human-to-bot conversations (abbreviations, casing, misspellings, morphological variants, paraphrases, punctuation and synonyms). On this test-suite, we show that common noise types substantially degrade the IC accuracy and SL F1 performance of state-of-the-art BERT-based IC/SL models. By leveraging cross-noise robustness transfer, i.e. training on one noise type to improve robustness on another noise type, we design aggregate data-augmentation approaches that increase the model performance across all seven noise types by $+10.8\%$ for IC accuracy and $+15$ points for SL F1 on average. To the best of our knowledge, this is the first work to present a single IC/SL model that is robust to a wide range of noise phenomena."
}