<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="yuan-etal-2020-cold">
    <titleInfo>
        <title>Cold-start Active Learning through Self-supervised Language Modeling</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Michelle</namePart>
        <namePart type="family">Yuan</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Hsuan-Tien</namePart>
        <namePart type="family">Lin</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Jordan</namePart>
        <namePart type="family">Boyd-Graber</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2020-nov</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)</title>
        </titleInfo>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Online</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Active learning strives to reduce annotation costs by choosing the most critical examples to label. Typically, the active learning strategy is contingent on the classification model. For instance, uncertainty sampling depends on poorly calibrated model confidence scores. In the cold-start setting, active learning is impractical because of model instability and data scarcity. Fortunately, modern NLP provides an additional source of information: pre-trained language models. The pre-training loss can find examples that surprise the model and should be labeled for efficient fine-tuning. Therefore, we treat the language modeling loss as a proxy for classification uncertainty. With BERT, we develop a simple strategy based on the masked language modeling loss that minimizes labeling costs for text classification. Compared to other baselines, our approach reaches higher accuracy within less sampling iterations and computation time.</abstract>
    <identifier type="citekey">yuan-etal-2020-cold</identifier>
    <identifier type="doi">10.18653/v1/2020.emnlp-main.637</identifier>
    <location>
        <url>https://aclanthology.org/2020.emnlp-main.637</url>
    </location>
    <part>
        <date>2020-nov</date>
        <extent unit="page">
            <start>7935</start>
            <end>7948</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Cold-start Active Learning through Self-supervised Language Modeling
%A Yuan, Michelle
%A Lin, Hsuan-Tien
%A Boyd-Graber, Jordan
%S Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)
%D 2020
%8 nov
%I Association for Computational Linguistics
%C Online
%F yuan-etal-2020-cold
%X Active learning strives to reduce annotation costs by choosing the most critical examples to label. Typically, the active learning strategy is contingent on the classification model. For instance, uncertainty sampling depends on poorly calibrated model confidence scores. In the cold-start setting, active learning is impractical because of model instability and data scarcity. Fortunately, modern NLP provides an additional source of information: pre-trained language models. The pre-training loss can find examples that surprise the model and should be labeled for efficient fine-tuning. Therefore, we treat the language modeling loss as a proxy for classification uncertainty. With BERT, we develop a simple strategy based on the masked language modeling loss that minimizes labeling costs for text classification. Compared to other baselines, our approach reaches higher accuracy within less sampling iterations and computation time.
%R 10.18653/v1/2020.emnlp-main.637
%U https://aclanthology.org/2020.emnlp-main.637
%U https://doi.org/10.18653/v1/2020.emnlp-main.637
%P 7935-7948

Markdown (Informal)

[Cold-start Active Learning through Self-supervised Language Modeling](https://aclanthology.org/2020.emnlp-main.637) (Yuan et al., EMNLP 2020)

• Cold-start Active Learning through Self-supervised Language Modeling (Yuan et al., EMNLP 2020)

ACL

• Michelle Yuan, Hsuan-Tien Lin, and Jordan Boyd-Graber. 2020. Cold-start Active Learning through Self-supervised Language Modeling. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7935–7948, Online. Association for Computational Linguistics.