@inproceedings{golovneva-etal-2022-task,
    title = "Task-driven augmented data evaluation",
    author = "Golovneva, Olga  and
      Wei, Pan  and
      Abboud, Khadige  and
      Peris, Charith  and
      Tan, Lizhen  and
      Yu, Haiyang",
    editor = "Bosselut, Antoine  and
      Chandu, Khyathi  and
      Dhole, Kaustubh  and
      Gangal, Varun  and
      Gehrmann, Sebastian  and
      Jernite, Yacine  and
      Novikova, Jekaterina  and
      Perez-Beltrachini, Laura",
    booktitle = "Proceedings of the 2nd Workshop on Natural Language Generation, Evaluation, and Metrics (GEM)",
    month = dec,
    year = "2022",
    address = "Abu Dhabi, United Arab Emirates (Hybrid)",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2022.gem-1.2",
    doi = "10.18653/v1/2022.gem-1.2",
    pages = "18--25",
    abstract = "In the area of data augmentation research, the main focus to date has been on the improvement of the generation models, while the examination and improvements to synthetic data evaluation methods remains less explored. In our work, we explore a number of sentence similarity measures in the context of data generation filtering, and evaluate their impact on the performance of the targeted Natural Language Understanding problem on the example of the intent classification and named entity recognition tasks. Our experiments on ATIS dataset show that the right choice of filtering technique can bring up to 33{\%} in sentence accuracy improvement for targeted underrepresented intents.",
}