<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="xin-etal-2018-batch">
    <titleInfo>
        <title>Batch IS NOT Heavy: Learning Word Representations From All Samples</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Xin</namePart>
        <namePart type="family">Xin</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Fajie</namePart>
        <namePart type="family">Yuan</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Xiangnan</namePart>
        <namePart type="family">He</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Joemon</namePart>
        <namePart type="given">M</namePart>
        <namePart type="family">Jose</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2018-jul</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)</title>
        </titleInfo>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Melbourne, Australia</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Stochastic Gradient Descent (SGD) with negative sampling is the most prevalent approach to learn word representations. However, it is known that sampling methods are biased especially when the sampling distribution deviates from the true data distribution. Besides, SGD suffers from dramatic fluctuation due to the one-sample learning scheme. In this work, we propose AllVec that uses batch gradient learning to generate word representations from all training samples. Remarkably, the time complexity of AllVec remains at the same level as SGD, being determined by the number of positive samples rather than all samples. We evaluate AllVec on several benchmark tasks. Experiments show that AllVec outperforms sampling-based SGD methods with comparable efficiency, especially for small training corpora.</abstract>
    <identifier type="citekey">xin-etal-2018-batch</identifier>
    <identifier type="doi">10.18653/v1/P18-1172</identifier>
    <location>
        <url>https://aclanthology.org/P18-1172</url>
    </location>
    <part>
        <date>2018-jul</date>
        <extent unit="page">
            <start>1853</start>
            <end>1862</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Batch IS NOT Heavy: Learning Word Representations From All Samples
%A Xin, Xin
%A Yuan, Fajie
%A He, Xiangnan
%A Jose, Joemon M.
%S Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
%D 2018
%8 jul
%I Association for Computational Linguistics
%C Melbourne, Australia
%F xin-etal-2018-batch
%X Stochastic Gradient Descent (SGD) with negative sampling is the most prevalent approach to learn word representations. However, it is known that sampling methods are biased especially when the sampling distribution deviates from the true data distribution. Besides, SGD suffers from dramatic fluctuation due to the one-sample learning scheme. In this work, we propose AllVec that uses batch gradient learning to generate word representations from all training samples. Remarkably, the time complexity of AllVec remains at the same level as SGD, being determined by the number of positive samples rather than all samples. We evaluate AllVec on several benchmark tasks. Experiments show that AllVec outperforms sampling-based SGD methods with comparable efficiency, especially for small training corpora.
%R 10.18653/v1/P18-1172
%U https://aclanthology.org/P18-1172
%U https://doi.org/10.18653/v1/P18-1172
%P 1853-1862

Markdown (Informal)

[Batch IS NOT Heavy: Learning Word Representations From All Samples](https://aclanthology.org/P18-1172) (Xin et al., ACL 2018)

• Batch IS NOT Heavy: Learning Word Representations From All Samples (Xin et al., ACL 2018)

ACL

• Xin Xin, Fajie Yuan, Xiangnan He, and Joemon M. Jose. 2018. Batch IS NOT Heavy: Learning Word Representations From All Samples. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1853–1862, Melbourne, Australia. Association for Computational Linguistics.