<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="yoo-qi-2021-towards-improving">
    <titleInfo>
        <title>Towards Improving Adversarial Training of NLP Models</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Jin</namePart>
        <namePart type="given">Yong</namePart>
        <namePart type="family">Yoo</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Yanjun</namePart>
        <namePart type="family">Qi</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2021-nov</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: EMNLP 2021</title>
        </titleInfo>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Punta Cana, Dominican Republic</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Adversarial training, a method for learning robust deep neural networks, constructs adversarial examples during training. However, recent methods for generating NLP adversarial examples involve combinatorial search and expensive sentence encoders for constraining the generated instances. As a result, it remains challenging to use vanilla adversarial training to improve NLP models’ performance, and the benefits are mainly uninvestigated. This paper proposes a simple and improved vanilla adversarial training process for NLP models, which we name Attacking to Training (A2T). The core part of A2T is a new and cheaper word substitution attack optimized for vanilla adversarial training. We use A2T to train BERT and RoBERTa models on IMDB, Rotten Tomatoes, Yelp, and SNLI datasets. Our results empirically show that it is possible to train robust NLP models using a much cheaper adversary. We demonstrate that vanilla adversarial training with A2T can improve an NLP model’s robustness to the attack it was originally trained with and also defend the model against other types of word substitution attacks. Furthermore, we show that A2T can improve NLP models’ standard accuracy, cross-domain generalization, and interpretability.</abstract>
    <identifier type="citekey">yoo-qi-2021-towards-improving</identifier>
    <identifier type="doi">10.18653/v1/2021.findings-emnlp.81</identifier>
    <location>
        <url>https://aclanthology.org/2021.findings-emnlp.81</url>
    </location>
    <part>
        <date>2021-nov</date>
        <extent unit="page">
            <start>945</start>
            <end>956</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Towards Improving Adversarial Training of NLP Models
%A Yoo, Jin Yong
%A Qi, Yanjun
%S Findings of the Association for Computational Linguistics: EMNLP 2021
%D 2021
%8 nov
%I Association for Computational Linguistics
%C Punta Cana, Dominican Republic
%F yoo-qi-2021-towards-improving
%X Adversarial training, a method for learning robust deep neural networks, constructs adversarial examples during training. However, recent methods for generating NLP adversarial examples involve combinatorial search and expensive sentence encoders for constraining the generated instances. As a result, it remains challenging to use vanilla adversarial training to improve NLP models’ performance, and the benefits are mainly uninvestigated. This paper proposes a simple and improved vanilla adversarial training process for NLP models, which we name Attacking to Training (A2T). The core part of A2T is a new and cheaper word substitution attack optimized for vanilla adversarial training. We use A2T to train BERT and RoBERTa models on IMDB, Rotten Tomatoes, Yelp, and SNLI datasets. Our results empirically show that it is possible to train robust NLP models using a much cheaper adversary. We demonstrate that vanilla adversarial training with A2T can improve an NLP model’s robustness to the attack it was originally trained with and also defend the model against other types of word substitution attacks. Furthermore, we show that A2T can improve NLP models’ standard accuracy, cross-domain generalization, and interpretability.
%R 10.18653/v1/2021.findings-emnlp.81
%U https://aclanthology.org/2021.findings-emnlp.81
%U https://doi.org/10.18653/v1/2021.findings-emnlp.81
%P 945-956

Markdown (Informal)

[Towards Improving Adversarial Training of NLP Models](https://aclanthology.org/2021.findings-emnlp.81) (Yoo & Qi, Findings 2021)

• Towards Improving Adversarial Training of NLP Models (Yoo & Qi, Findings 2021)

ACL

• Jin Yong Yoo and Yanjun Qi. 2021. Towards Improving Adversarial Training of NLP Models. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 945–956, Punta Cana, Dominican Republic. Association for Computational Linguistics.